Numerous software companies have since been born eager to offer a spectrum of products ranging from settlements systems to load forecasting tools. So called ‘out-of-the-box’ software packages can be purchased to shorten the application development cycle. Instead of building everything from the ground up, existing components can be pieced together or an entire system can be bought. Today, this is typically the case. Rarely do software build efforts start completely from scratch. There is no doubt that use of software packages in energy software development is compelling. To be beneficial however, the total cost of implementing the software package (licenses, fees, configuration, maintenance, etc.) should be less than the cost of building the equivalent functionality. Unfortunately, this realization is not easily achieved for many reasons.

Evolving Business Rules
Much of the success of implementing a software package hinges on a single question; how well does the software package meet the requirements of the business need? The answer is quite elusive because accurately defining business requirements is one of the most difficult parts of engineering a software system. To begin with, systems that manage wholesale and retail electric markets conduct complex business processes. Webs of software systems are interwoven to procure wholesale market reserves, maintain reliable power operations, aggregate and profile every drop of electricity in each region, settle the market at specific time intervals, facilitate customer switching and billing, and finally store and provide market participants access to the huge amount of data. Now consider the dynamic landscape of the energy industry where business rules are fluid due to regulatory forces and changing market designs. With less than three years of full deregulated operation under its belt, Texas is considering a redesign of its wholesale electric market to manage congestion on a nodal rather than zonal basis. In other regions of the country, uncertainty surrounding FERC’s Standard Market Design provides shaky foundation. Each region offers unique design issues for the software engineer. But, such is the reality of our industry. Nobody said innovation was easy. Forced to base decisions on obscure and evolving market rules, utility and energy IT managers and designers must follow the notion that ‘close counts’ when selecting a software package. Therefore, choosing an incompatible software package can be an easy mistake to make.

Anatomy of Packaged Software
Make no mistake. Software companies create ‘out-of-the-box’ packages for one reason, to make money. In most cases, the goal is to build a product that can be resold to multiple customers. A base architecture and data schema act as the static core of the package. The core generally will meet a basic demand in the business world such as customer relationship management, statistical analysis, or account billing. Providing a generalized component, the core usually doesn’t change between implementations at different customers. Sitting on top of the core is a configuration layer. The configuration layer provides the malleability to resell the product. The unique requirements of each customer are programmed here. Again, the design is for profit, which doesn’t always equate to efficiency for each customer.

Unused Components
Business processes vary by customer and it is possible to have bulky pieces in the base architecture or data schema that are not needed (or wanted) but must be bought. These inane components may even interfere with existing functions if not activated. For example, a company just purchased a flashy, web-enabled billing package. Embedded in the core architecture is a component that provides general ledger functionality which the company does not plan to use. A different, more compatible general ledger system was bought. But during implementation, company developers realize the billing engine malfunctions unless the core general ledger component is used. The effort to circumvent the entrenched general ledger piece may double the project budget. Even without interfering, items like unused tables in the data schema still consume costly system resources and linger to confuse developers.

Not Built in a Vacuum
The difficulties of integrating software packages are not intuitive. Packages are not constructed in a vacuum. They make assumptions about the environment, interfaces, and architecture that surrounds them. Software vendors themselves may not be aware that these assumptions exist because not every scenario can be tested. Therefore, these assumptions, sometimes subtle, are hard to discover other than through experience and can be nearly impossible to alter by way of configuration. If the package was constructed on a Windows operating system, it may not be portable to Unix or it may respond completely different on a Unix platform than what was observed on Windows. The same can be said for database interaction and especially interfaces. Interfaces are where software packages make many assumptions and may have complicated requirements to transfer data. How does the package communicate with everything else? Can data cross the API interface in any format? Is the accompanying documentation clear, accurate, and complete? The application will undoubtedly require the data to be presented in a certain way for it to be read. But the product manual surely will leave out details about package nuances that are left for the customer to discover.

Black Box
Software companies employ various strategies to extract as much revenue as possible from their products. A popular method is to protect parts of the architecture and hide the internal workings from the customer. Any changes to these pieces of the code must go through the vendor. More than likely, modifications to these pieces will be required and vendors lock in years of maintenance dollars. Of course, competitors can swoop in and steal the concepts if nothing is proprietary so some of this is justified. But, software products do exist that give the customer full control over the code once purchased. Some questions to ask include: If the package does contain protected code, can the customer work around it if necessary? How about to avoid paying maintenance fees? Vendors might get upset if a crafty developer programs around a protected component. A voided contract with financial penalties or legal action may result.

Performance
Interaction with RTO and ISO entities create huge performance requirements for software such as settlement systems. The amount of data to be crunched, transferred, and stored for each trade date is imposing. We’re talking up to hundreds of thousands of database records processed per night for settlements starting with unaggregated load and generation. Nightly batch runs may even process multiple trade dates. Some protocols require systems to accommodate up to twelve settlement runs per night. Consider nodal settlement where there could be thousands of pricing points per trade date and the software system is quickly glaring at a mountain of data to spin through. Because packages are not necessarily built with efficiency for the individual customer in mind, performance may be impacted. Inefficient interaction with the database or other applications can make it expensive to reach performance requirements. If allowed by the vendor, poor performing components may need to be retuned or replaced. Otherwise, beefing up hardware may be the only option.

Hidden Costs
Besides the expected assortment of licenses and maintenance fees, estimating implementation costs accurately is a challenge. Depending on how complex the package is to configure and the quality of support from the vendor, configuration costs can increase without warning. If vendor assistance is necessary (i.e. black box changes), poor service and slow turnaround may be a significant impact. Upgrades are an expected part of buying software. What happens when the need to upgrade one application triggers a chain reaction where interfacing applications must be upgraded to satisfy compatibility? To keep a database supportable by the vendor, an inexpensive upgrade is required. But, an existing financial management application may not be compatible with the new database version. Another hidden cost is the Guinea pig effect. Try to not be the unsuspecting Guinea pig or beta tester for the software vendor. Attempting to use a product in a situation where it has never been used will act as a demonstration of entropy. Because an application can process ten thousand records in an hour doesn’t mean it can scale to one million records without a problem or additional architectural analysis. Unknowingly conducting product research is a recipe for snowballing costs due to time and resources. If possible, push the onus on the vendor to attempt new requirements with the package.

The Road Ahead?
New hopes for productivity in software engineering are being experimented with today. Outsourcing is currently being pitched as a solution. Sending build efforts across the country or overseas to take advantage of cheap labor will lower costs. The question is does quality decline with decreasing bill rates? The potential of outsourcing is even more suspect in the utility and energy industry. When business requirements are evolving as fast as they are today, being as close to the customer as possible helps reduce latency and facilitates accurate delivery. Then is it possible that our industry is trending back around, full circle, to customized builds? At least in this case, an unfamiliar product is not the focal point of the work and the system can be built fully to specific requirements from the architecture on up. Does controlling every piece of the puzzle and knowing the trapdoors eliminate enough maintenance cash burn downstream to justify the costs upfront that come with a customized build?

In any event, deploying packaged software may or may not yield the best productivity depending on the situation. But, expecting a software package to meet your needs perfectly is unrealistic. Finding the best fit and anticipating the problems should be the goal. Because underestimating or worse, not knowing, the pitfalls of installing ‘out-of-the-box’ software packages can liken an implementation to pounding a square peg into a round hole.

To subscribe or visit this site go to:  http://www.energypulse.net

Copyright 2004 CyberTech, Inc.