Tag Archives: trends

Software Development Trends and Predictions for 2011

Share the article!

Oh how time flies! I’ve been quite remiss in providing my yearly predictions in the Software development space. Must be the mini economic depression that we’ve gone through that just doesn’t appear to be ending for many folks. I am surprised myself to find the last time I published my predictions was in 2007 That’s a year before the epic market crash.

Well it does turn out that most of my predictions (or rather observations) came out to fruition. That is,

Virtualization, Cloud Computing, Dynamic Languages, Javascript Browser Integration, RIA frameworks, openID, the death of Webservices and life after EJB are all common features of today’s software development landscape.

What is surprising is how little these trends have changed in the last 4 years since I made them. Appears to be just steady improvements (or declines) on a year to year basis. What is interesting to note is that when a concept or technology has a declining, it does persist. The only question is the velocity of the decline. Take for example WS-* Standards and EJB, over the past 4 years progress in this space has slowed to a crawl (EJB) or halted completely (WS-*). Of course these spaces are not entirely dead economically. Massive amounts of investments have been made in these two specific areas and their certainly is a bias and vested agenda to continue to pursue. These are today’s generation’s COBOL legacy apps.

My other 2007 predictions have had made less of a discernible progress. Parallel Programming ( at the CPU level) and Semantic Web. There of course have been massive progress in the CUDA space and in LinkedData, but these technologies are not as ubiquitous as I had predicted.

There however is one prediction that is experiencing a trend reversal. JCR in the years following 2007 made robust progress in that it was common to see many new Java projects based on it. However in recent years, likely because of competition with NoSQL, there seems to be a compelling alternative to document storage.

Despite the economy (which I attribute more to global equilibrium), these are very exciting times for software development. There is a sea change that is happening that will rapidly change the face of the industry. Here are my 2011 predictions for software development.

  1. Enterprise AppStores – Okay, folks have been pointing this out for years using different names ( Widgets, Gadgets, Midlets etc) I mean Java itself got its boost because of Applets. We now know what has come of applets. The Thin Client revolution was hyped a decade ago. Well it turns out, it has arrived, courtesy of faster and more energy efficient processors and gesture driven devices. Enterprise will demand the kind of apps they find on their smart phone and tablets. Gone will be monolithic apps with loads of features that are rarely used. In will be highly specialized applications with high usability. Corporations, the control freaks that they are, will insist on governance. This ultimately implies an AppStore per enterprise basis.

  2. Javascript Virtual Machines – This trend has becoming bigger and bigger every year and as much as we find the language and environment to be difficult and unscalable, the community will have to deal with it. A majority of new development projects will need to find out how they can scale Javascript development or die trying. There are several competing alternatives, these include Javascript frameworks like jQuery and YUI, treat Javascript as assembly language like GWT and CoffeeScript and pretend it doesn’t exist like JSF, Vaadin and RAP. Your poison would like be chosen based on the kind of app you are building. What we will see is a major percentage of the app would be executed in a Javascript container, whether that be in the browser, the mobile app (PhoneGap) and the server (see: NodeJS, Jaxer).

  3. NoSQL – This is a harder call to make simply because not many applications are going to need this kind of scalability. That is if you look at it from the point of view of large monolithic applications. However, if an enterprise is to serve thousand of little applications, they better quickly figure our a way to manage that proliferation without the conventional overhead of managing the typical enterprise app. Virtualization gets you halfway their, but cloud based infrastructure, particularly storage managed by specialized NoSQL servers would be the economical way to go. The enterprise will consist of horizontally scalable ReSTful accessible endpoints that will be spewing HTML and Javascript. End users will be quickly able to compose apps and install them on their own devices.

  4. Semantic Indexing – So now that you have these massively horizonally scalable data stores, you are faced with an even bigger problem of finding anything. Full text engines like Lucene/Solr will be standard fare, however the Enterprise will demand higher fidelity in their searches and this will drive demand for indexing beyond inverted indexes. Expect to see servers with hundreds of gigabytes of DRAM to enable this.

  5. Big Analytics BigData will need analysis. Analysis is only as good as the quality of your data. There will be a rising demand for higher quality data sources, data quality tooling and of course alternative ways to analyzing and visualizing data. There will be intense debates between Map-Reduce NoSQL and traditional OLAP vendors. NoSQL will need to progress quickly enough before customers realize their inadequacies. Look to Hadoop to increase its dominance in this area.

  6. Private Clouds – Despite the economic and technological incentives of outsourcing one’s infrastructure, there will always be that organizational and human need to build and own one. Private clouds of course will overlap and need to integrate with public clouds, simply because the dominant business model will be SaaS. Look for OpenStack to emerge as the clear leader.

  7. OAuth in the Enterprise – OAuth is already widely used in consumer applications. With OAuth 2.0, I expect rapid adoption in the enterprise space simply because there is no other viable alternative. Application coordination will be a necessity and OAuth will be its enabler in secure environments.

  8. Inversion of Desktop Services – Services that we commonly expect on a desktop like the file systems, short cuts, contacts, calendars and programs will increasingly reside in the cloud. This is a consequence for the need for greater security on mobile devices and the need to share information among muliple devices and muliple collaborators. The filesystem of the future will be collaborative shared spaces. We are already seeing Services like Dropbox, Instapaper, Evernote, Springboard, Delicious and Twitter serving as the means of coordination between mobile applications.
  9. Proactive Agents – For decades people have been forecasting the emergence of digital personal assistants that would actively react to the environment on one’s own behalf. The emergence of always present smart mobile devices and cloud computing shared spaces will be the catalyst for the developing of active agent based computing platforms and frameworks. At present, most computing is merely reactive, that is servicing web requests only on command of a user. Future computing will include a proactive aspect that suggests courses of actions to users. Semantic technologies like Zemanta and OpenCalais provide intelligence to writers by suggesting tags that are relevant to a written document.

  10. Migration to Scala – My hunch on the programming language that will have the highest growth in adoption and interest would be Scala. Scala’s appeal is that it is elegant and can express complex constructs succinctly. I see most adopters migrating from serverside developers looking for a new shiny toy to play with. Scala frameworks like Lift and Akka, and IDEs like IntelliJ will be the wind that propels this migration. Alternative JVM languages like Groovy, Clojure and JRuby will likely plateau in popularity as a consequence.

What now is becoming apparent with these trends is greater fragmentation in IT. No more is one kind of database, one kind of programming language, one kind of operating system, one kind of UI framework going to be viable. One cannot control anymore what its built on (specifically for any cloud based service), however one may still have control of the data (see: Dataportability.org) and this will be key.

Share the article!

Top Five Java Technologies that Didn’t Make the List and Why

Share the article!

I’m delighted that Joseph Ottinger has so graciously posted my previous post “Top Five Java Technologies to Learn in 2008” on the Serverside. A list limited to a select five entires surely will bring out some controversy. There will always be those who would feel left out. Of course, the list could have been expanded to 10 entries, but that defeats the entire point. As the Attention Economy has revealed, an individual’s attention is a scarce commodity.

Its been suggested that Polyglot programming be in the list. Even though I do subscribe to the notion that learning other languages are beneficial to one’s craft, it simply is not pragmatic advice. It is not practical to recommend that someone study Ruby, Groovy, Scala and who knows what other language is vying for your attention. Stick to a couple of languages and do it well. Some languages are better than others for certain tasks. However the biggest fallacy of all is that, a dynamic language is not considerably better than a static one. It’s no magic bullet.

Finally, of the Five technologies I selected, there’s an underlying recipe to it. That is, all are about infrastructure. See, it isn’t about the language. If anything is going to make you successful in creating the next killer it app, then the platform that it runs on would influence your success more than your selection of language. As anecdotal evidence, look at the Chandler project, they had to re-invent the wheel several and in fact did that several times. Life would have been simpler if they used what ever was out there. Unfortunately, NIH reared its ugly head before it was too late to notice.

Let me go down the list to explain my point about infrastructure. OSGi is about managing the complexity of deployment. The default Java classloading mechanisms are clearly inadequate for a system where multiple versions of the same jar file are used. A higher level structuring mechanism is required and that is what OSGi precisely provides. Bill Kayser has a very good read on this matter:

In most cases, the JVM still loads classes from the big vat of class soup indicated by the class path. What is needed is something more fundamental, something that would completely xe2x80x9cinvertxe2x80x9d the control of the whole application, executing everything as services, not the chosen few components.

Java Content Repository (JCR) is about managing unstructured content. In my development travels I find it disappointing to find that most projects consider a RDBMS as the only choice of data storage. It is clearly unfortunate since almost all systems require some kind of unstructured content storage. Even though you could shoehorn your data in blobs in a RDBMS, there still remains the additional task of building infrastructure around it. For example, how would you search through those blobs? SQL’s text search mechanisms are going to be a massive disappointment for those who are all too familiar with Google. But going beyond just usability, there will always be the need for users to annotate the data they work with.

GWT is about managing the complexity of Javascript. The two main problems with Javascript are (1) the extreme flexibility of the language and (2) the lack of consistency of browser implementations. GWT tackles the first problem by employing Java and treating Javascript as an assembly language. For most developers, it should not be necessary to have to delve deeply into assembly language to get your tasks done. The second problem is handled by the maintainers of GWT, they have painstakingly crafted variants of the Javascript that is generated to handle the different nuisances of the browsers. As browsers continue to evolve, GWT developers will continue to tweak their compiler. We should all be thankful that someone else is doing this arduous work.

Then there’s the controversial selection of Groovy. Groovy allows you easy access to meta-programming features in a dynamic manner. Java infrastructure has continually improved its meta-programming features. It has progressed initially form JavaBeans then to BCEL/ASM (runtime bytecode management)to finally Annotations and Aspect Oriented Programming. Groovy’s meta-programming features is one additional method in the tool chest. Of course, the scripting language shortcuts are indeed helpful.

Finally there is that notion of cloud computing. Cloud computing has been around for some time now, however I think Amazon’s entry into the space with a pay as you go model is truly disruptive. It is the convenience of it all that makes it all different. The amount of time saved, by not having to do it yourself or even not having to require a conversation with a human to get it done, is tremendous. The ability to scale up without a massive up front investment is changing the playing field as we speak.

In summary, the list is about technologies that add a base infrastructure to build you next killer app. Surely you can use whatever framework is out there, but ultimately your success will be influenced based on how much work you didn’t have to do by leveraging what was already out there.

Now going to the top five technologies that did not (or just barely) make the list:

  • #5 Android – Google’s attempt to turn the telco space on its head. This is expected to be not only game changing but disruptive. I of course will like to wait for (a) an actual hardware device to play with (b) Google winning the 700Mhz spectrum and (c) the deployment of 700Mhz cell towers across the states. The problem I have with Android is the problem the plagues J2ME devices. How do I keep my sanity as the variety of mobile devices increases exponentially? I have yet to have heard a compelling story (OSGi perhaps?) in this area. That’s plain unfortunate because it’s the biggest pain point for mobile developers.
  • #4 JRuby – Rails is certainly getting a lot of attention; JRuby channels that attention towards the JVM. The Ruby job market has grown to a stunning 3.3% of the size of the Java market. If you need something new (don’t ever try this with legacy code) then possibly the quickest route to a working application is via Rails (use Streamlined while your at it). However, not all applications consist only of CRUD screens, you’ll need to add more mojo into it. That’s when you have to question Rails viability. Convention over configuration only works well if you’re building something conventional.
  • #3 Scala – Scala is a brilliant well thought out language. It’s a language that is not only extremely expressive but one that doesn’t have the performance baggage of a dynamically typed language. In short, you could use it to build bleeding edge infrastructure stuff. The kind of stuff you would never dream of building using slow as molasses languages like Ruby. Sure, you have to subscribe to the static typing religion, it’s an unfortunate trade-off you have to make if you want to build really tight systems. It however didn’t make the list because I would only recommend it to ‘rocket scientists’ building serious heavy lifting infrastructure. Web front end developers need not apply.
  • #2 Seam – Gavin King (Hibernate’s creator) is an accomplished developer. So when he builds another framework, it behooves you to take notice. Seam is his latest creation and it merges JSF and EJB3 into a seamless workable framework that’s all hangs together with Annotations. Unfortunately, its strength is also its weak spot. All that annotation work is going to be a nightmare to debug. Personally, I think Spring is good enough, however if you must have an overarching framework then Seam would be it.
  • #1 Flex – If anything should have been on the list, this would be it. If you want to build rich applications on a browser then plain DHTML and Javascript isn’t going to cut it. The most innovative desktop like applications on the web are unfortunately built on Flash. GWT isn’t going to help you build applications like YouTube, Mindomo, Buzzword, Sketchcast or Pandora but Flex will. The bad news is that Flex isn’t Java, however the good news is that Adobe’s soon to be open sourced server (BlazeDS) is thankfully in Java.

So if you do have that extra bandwidth, I would recommend learning about these additional five technologies. Who just never know, these just might be the top five for 2009 ;-) .

Share the article!

Software Development Trends for 2007

Share the article!

Okay, time to start my yearly ritual of predicting the future of the coming year. I’ve attempted to do so for the last two years, 2005 predictions and 2006 trends. You be the judge on how successful I am on this activity. Not that I’m trying to claim to be clairvoyant, but rather by taking time to write about it I get a better grasp of what’s emerging and if I’m lucky enough spot those ones that surprising one’s that people may miss.

  1. Virtualization – Affordable multi-core systems and muli-cpu systems and the ever growing costs of maintaining racks of servers would push operational staff to deploy more virtualization solutions like VMWare and Xen. Vendors will begin to package their server solutions as virtual appliances. Java solution vendors despite being O.S. agnostic will also adopt this approach. Software vendors will follow Zeus ZXTM VA and re-work their software to be usable within a virtualized environment. The JVM will be also rebundled as a VMWare virtual machine, in fact the opensourcing of the JVM will have its greatest impact on this area. Update: See BEA LiquidVM.
  2. Cloud Architecture – The cost and maintenance benefits of services like Amazon’s S3 and Elastic Cloud will have such a tremendous architectural appeal that more and more services will be architected in a similar way. Furthermore applications will be re-architected to make best use of these services. There will be several roadblocks along that will limit its uptake however this would be a long term architectural trend.
  3. Parallel Programming – Developers would need to scratch an itch as to how best to develop systems to support multi-core hardware. Intense efforts will be attempted at extracting parallelism at the micro-level. There will be a resurgence of old ideas like dataflow computing, implicitly parallel functional languages and stream processing, however this would be of only niche utility, primarily useful for the gaming and scientific communities. Coarse grain parellism in the form of virtualization and to a lesser extent cloud architectures will continue to dominate. Personal users will begin to realize the benefits of mega-tasking, and the majority of demand for muli-core/cpu machines will spring from this usage.
  4. Dynamic Programming Languages – Ruby (and Groovy) will continue to gain converts, however it will be confined to a minority of highly capable development organizations. The ease of creating Domain Specific Languages (DSLs) in Ruby/Groovy will lead to rise in literature promoting the idea. Tools for Ruby/Groovy will continue to develop in sophistication. In the end, Ruby and Groovy will become an adjunct tool for Java developers, just like Ant. The .NET IronPython implementation will continue to be superior to the Java based JRuby and Groovy implementations, however it will gain less mind share on purely subjective and social reasons. Python doesn’t have that ‘X-factor’ and .NET already has a de-facto ‘scripting’ language. Update: Groovy finally released version 1.0.
  5. Java Content Repository – Organizations will begin to worry about their creaky poorly supported CMS and begin towards safeguarding their content assets by migrating to a JCR implementation. This will further be driven by the emergence of more sophisticated applications that work off the JCR API.
  6. The Browser Javascript Integration Platform – People will be utterly stunned by the utility of browser based extensions as a means of integration. The developments such as Mozilla’s Operator, robust AJAX libraries like Tibco GI and Social Web applications like PiggyBank will lead to an explosion of desktop application develpment that would be solely browser based. Mashups will be taken to the next level where workflow integration would be performed at the browser. Javascript performance will be improved with JIT compilation. Update: First glimpses of Firefox 3.0 revealed as ‘Information Broker‘.
  7. Eclipse Ubiquity – Eclipse development tools will be prevalent in development organizations regardless of programming language chosen (i.e. C++, Flash, Javascript, Ruby, Python, etc), however, the growth of RCP based applications will be challenged by Browser Based Integration techniques. What’s more likely is that Eclipse plugins to support the competing browser based technology will be developed. Eclipse based Enterprisey toolsets catering to project lifecycle management and model driven architecture will be commonplace in 2007.
  8. Multimodal AJAX frameworks – Interest in Google’s GWT toolkit will continue to grow, particular in more traditional Java based organizations. GWT will be used not only to develop AJAX applications, but Mobile based applications using J2ME. This will be at the expense of Laszlo that will make excruciating slow progress to finish its AJAX and J2ME bindings. Tibco GI would make strong headway in traditional development organizations, however one would see little uptake in this technology in Web 2.0 offerings. Tibco GI’s traditional desktop like UI despite being extremely impressive doesn’t fit the low cognitive overhead requirements of Web 2.0 applications. Until it’s refactored to allow bite sized configurations, I expect to see it used more in Browser based extensions. Update: Laszlo release their first beta that supports multiple runtimes (i.e. flash and dhtml).
  9. Life after EJB – The EJB nightmare is finally over. Unfortunately, the world we’re transitioning to is extremely fragmented. Out of the carnage, Spring and JBoss (i.e. Hibernate, jBPM, Rules) have emerged as the two main competitors in the enterprise space. Spring and JBoss are actually meta-frameworks, Spring structures participation using code, JBoss does it through employment. Other vendors will align with one or them or both (see: Alfresco, Nuxeo, Atlasian etc). However, it is safe to say, that any respectable enterprise java worth its salt would be based on either of these two frameworks. JBoss Seam framework will gain traction and JSF will be the new Struts.
  10. Decentralized Identity Management – Every architecture group in big IT organizations has grapled with the problem of unified identity management across their enterprise. Unfortuntately, it has been a complex problem from both the technical and social perspective. Fortunately, a viable and lightweight solution has been developed (i.e. OpenID). I expect OpenID to take tremendous mindshare in 2007, in an order of magnitude greater than other solutions such as SAML, Liberty and MS Passport that came before it. Update: Emre Sokullu writes that Wikipedia is expected to support OpenID.
  11. Dead WebServices – WS-* and its corresponding specifications like SOAP and WSDL will be declared dead by year end. You’ll find the decalaration first by tool vendors and open source projects that’ll refuse to expend anymore effort treking the wrong path. The ensuing void will force SOA to drift to be more about methodologies and SOA ‘governance’ (i.e. SOA 2.0 ).
  12. A Little Bit of Semantics Everywhere – Jim Hendler summarizes this observation best. The top down ivory semantic web approach has run out of steam, fortunately lots of grass roots developments like del.icio.us tagging, revyu, microformats, google base, flickr etc. are providing developers with a massive (albeit less structured) knowledge base that will in 2007 springboard semantic web technologies into the spot light.

There’s probably a lot of other trends that I may have missed, so please let me know.

Share the article!

Top 10 Emerging Software Development Trends for 2006

Share the article!

It’s a bit too late to provide 2006 predictions. My problem was that in the begining of this year, I couldn’t make heads or tails as to what was emerging in the software development world. There simply was too much going on, and it really does take a while for everything to sink in. Now that I do have my bearings, I bring to you my top 10 emerging software development trends circa 2006.

  1. Virtual Applicances – Ever since VMWare releasd its player and server for free, there’s been a groundswell of virtual appliances being built. These applications have the entire software stack including the operating system bundled as a VMWare virtual machine. Rather than worry about the complexities of building an installer to conform to an endless list of customer user environments. These virtual machines are pre-optimized disk images of an actually tested envrionment. Installation costs take out a huge chunk of the overall cost of deploying software, take this to zero and you are in good shape to contain your dwindling software margins.
  2. Virtual Ofice Spaces – With all the Web 2.0 application emerging, add to the mix the more powerful tools available for distributed development and you have a platform that makes it increasingly unnecessary to even maintain an office. Consider for now that talent and resources are spread out all over the world. To remain competitive, one needs to effectively leverage distributed teams. The tools to that are dirt cheap and do expect more people to be spending their work week in their local Starbucks and Kinkos.
  3. Multimodal Portals/Widgets – So now that we aren’t even in the offices, and walking around the park instead, I can’t really tell what kind of device I’ll be lugging around at any given moment. So when I need to jot down an important thought, work out an urgent task, schedule a new appointment, enjoy some media, I would like that piece of information available to any device that I happened to be carrying. Expect to see more Apps that make it easy to embed themselves in more kinds of portals. Expect to see portals that reside in multi kinds of devices.
  4. Portable Social Networks – Social networks are really big, MySpace is an example of one that those humongous entities that is bordering on scaring. But let’s be real, as these people grow up, the context where they live their lives becomes more varied. It’s simply inevitable that one has social networks that are separate from each other. In addition, one interface can’t fit all, the structured interfaces of linkedin don’t play too well in the chaotic environment of MySpace. These networks are going to be extremely valuable over time, expect a lot of tools devoted to making your social network more portable.
  5. Participatory Media – Blogging, Flickr and now YouTube. More and more people are sharing their media, from text, to photos and now video. Expect more kinds of media being shared, and expect more mashups/mixins. Expect to see Web 2.0 applications to support this demand. In addition, expect more content originating from your own social circle.
  6. Voice Enhanced Apps – If you’ve been in a conversation with a TellMe voice application you would appreciate how much this field has progressed in the last decade. Let’s get real, not everyone can ‘text’ without looking at their keypad. In fact, a lot of these background widgets that reside on your phone are making driving extremely dangerous. With American spending more and more time on the road, expect more people to use voice as the medium of interaction. There’s a good reason why podcasts like ITConversations are so popular. I couldn’t figure out why, that’s simply because I have a 5 minute commute to work. But for the rest of the world, well reality is, they’ve got their hand on the wheel and eyes on the road most of the time.
  7. Information Diet Programs – So now that we are bombarded with information and media from all sources and all our social networks, what then will keep us from becoming overloaded? We need applications to better manage how we respond to the constant barrage of information demanding our attention. Although we can always choose to shut it all off, however I suspect we can’t. Like any bad habit, we’ll need an effective ‘diet program’ to help manage our intake.
  8. Peer to Peer Monitoring Systems – As we become dependent on portals, widgets and mashups, essentially distributed applications with multiple owners, something needs to ensure that they all are working together in harmony. The current management and monitoring systems of today are inadequate for the task, too complex to setup and inflexible to maintain. What is needed are more dynamic monitoring systems that can slice through networks.
  9. Lightweight Non-Ruby Web Frameworks – People want their applications yesterday. Therefore developers need to take the shortest path to a solution. Yes, we all want to become quick, however we want to avoid being dirty. So new web frameworks, taking their queue from Rails, will emerge to provide solutions. See, not everyone can get a gig programming in Ruby. So these frameworks will eschew traditional standard frameworks and focus more on what works and what works quickly (see: rife, trails, grails and trimpath ) at the sametime preserving existing technoloical investment.
  10. Guilt Reduction Apps – Finally, despite all these attention consuming applications, there is always something that we continue put away for a later day. Procastinating endlessly, in the hope that we have more time in the future to focus on it. Well, over time the guilt builds up, and over time it becomes unbearable. Unlike background applications that are used constantly, these apps are use once applications. Applications that once your done, absolves you of the guilt of not doing it.

If you think I missed out on an obvious trend, then I do suggest you consult my 2005 list (“Ten Emerging Software Trends for 2005” and see if its there. If you don’t find it, then please make sure I know about it. I for one don’t want to miss out on the action!

P.S. I’m out of time, but I’ll post relevant links next time.

Share the article!