Interfaces required to multiple domains

I think their decision to keep themselves a software infrastructure company is smart. In this way, they can apply their systems to many market segments where operations are involved. When operations are performed, some kinds of data are generated and often times those data should be collected, stored, and analyzed to tune and improve operations and business processes. In order to dive into new domains, they need to keep adding new interfaces as well as adding and revising in areas they already cover. Dave Roberts told me that they now have close to 500 interfaces.

Coming from the IT segment, I see people tending to converge to a handful of well-defined standards and, therefore, interfaces. When I first put my foot into the data center market, I was very, very surprised to find out that there were many interfaces on the facilities side. Although BACnet is becoming a major force in the data center facilities protocol of choice, there are still several other protocols, such as Modbus and LonWorks, being used. An IT guy like me tends to think we can force facilities to adopt a single standard to consolidate all the protocols into one, which is IP. I now know it does not work that way. I got involved in NIST’s Smart Grid Interoperability Panel, which was organized to come up with a set of standards to allow smart grid to function without conflicting technologies and protocols. The power industry has been around longer than IT, and there are many standards by IEEE, IEC, and others. The power industry has been conducting business to keep the lights on for more than 100 years, and they will not listen to IT about consolidating everything to IT technologies and protocols, for sure.

How to translate domain specific requirements for software developers

OSIsoft maintains that their core PI system is generic and does not change when they apply PI to different vertical markets. When they pick a new domain, they add new interfaces specifically required for that domain. So every time they step into a new domain, they need to worry about yet more interfaces to maintain. This seems daunting, but it is the only practical way to have a generic system to apply to many areas, such as the power industry, oil and gas, and building management segments.

For each vertical domain there is a dedicatedindustry management team that includes experts in that field who can communicate natively with customers. The experts get agreements on requirements, then translate those requirements to a specification for software development teams and partners/ecos to work on.

How to enter a conservative industry like the power industry

IT’s change of pace is very fast. New technologies come and go quickly, sometimes within months, if not days. In contrast, utility companies are very conservative and do not replace their technologies and equipment for many years until new technologies or equipment are proven to work solidly. I was curious to find out how a software company like OSIsoft could penetrate into the conservative power industry. In the 1990s, OSIsoft partnered with Westinghouse and also with ABB. Through their introductions to utilities, they started to work with utility players. They expanded their market presence in the utilities market. Although there are a lot of similarities, each utility has specific needs, which triggers customization. But OSIsoft does not provide customization services. Customization is done by utilities themselves or system integrators. Nearly all—97%—of their revenue comes from software maintenance; the remaining 3% comes from basic services such as installation. So a highly configurable nature is important for their product.

Sharing data among multiple entities

In general, if two entities work together, it would be most beneficial to share data among the two. For example, let me refer to the power grid in California. California ISO (CAISO), which reliably balances power supply and demand on the transmission, does not maintain the transmission lines. The lines are maintained by PG&E, a local utility in my region that also is responsible for the distribution grid. Power imbalances can be caused by operational or equipment problems. Therefore, it is very useful if CAISO shares data with PG&E so that they can work together to solve the problem. For this, OSIsoft has released a new feature called PI Cloud Connect, which allows highly granular data to be shared with specific accessibility control in a cloud setting. In this way, any number of organizations can share time-series data with a specific access privilege. Yes, this is a good application of ICT.

Analytics

Once data are captured and stored, they are analyzed to derive useful information to improve operations and business processes. Analytics can be done at many levels. They can be as simple as out-of-bounds values analysis all the way up to prediction. Here OSIsoft does not do its own analytics packages but makes sure to plug in others’ packages seamlessly to the PI system. I am currently looking into analytics more in detail. Because analytics is a very broad term and it contains so many angles, most presentations or white papers on products do not mention it in detail. That is frustrating, to say the least.

What is an example of analytics in the utilities business?

Analytics example 1: equipment preventive maintenance

Do you see boxes of different colors and shapes on utility poles around you? One of those boxes is called a transformer and is used to step down high voltage to lower voltage before power gets to your home. Most transformers are based on the electromagnetism discipline and degrade physically as time goes by. If a transformer malfunctions or fails, power to your home will be interrupted. It would be nice to know when to repair or replace it before it fails. One of the analytics packages can monitor its health, bounce it with the historical trend, and provide an early warning.

Analytics example 2: wind power generation

Another example is in wind power generation. Wind is hard to predict. It is blowing one moment but not the next. It is vital to balance the demand and supply of power every second. If we cannot predict power generated by wind, it makes it more difficult to balance power. So it is very important to predict when wind blows and when it stops. Predictive analytics is used widely in weather forecasting, and wind prediction is part of it. First, a prediction model is developed from the historical data, and the model is fine-tuned and modified as more data are collected.

Analytics example 3: smart charging for EVs

Currently, in California, power demand increases as the day goes on and hits a peak in the early afternoon. It goes down to its lowest point during the night. An electric vehicle (EV) like the Nissan Leaf or Chevy Volt is known to draw about the same amount of power as a typical household. If they are charged when power demand is at peak, we run out of power to satisfy demand. But during the night, we usually have plenty of power available, and it is suitable to charge EVs at night at home. This is what a typical EV owner does now. As more public charging stations pop up, and faster yet power-hungry new charging technologies proliferate, charging may be done during peak time. That would disturb the power balance and lead to outages. For this reason, smart charging needs to be developed and deployed. The result of this type of analytics would dynamically allow charging to start when supply satisfies demand.

Different utilities could use an analytics package developed by one utility, but OSIsoft does not share particular users’ analytics algorithm with others. OSIsoft has its users communities, and those who belong to them might share such an algorithm via community. The T&D User Group community exists for 20 years, and they tend to share information when there is no competition among them.

Analytics example 4: more renewable energy sources for power generation in California

California has adopted a renewables portofolio system, known as RPS. This specifies the minimum percentage of renewable energy sources, like solar and wind, in power generation. California plans to attain 33% of all the power from renewable energy sources by 2020. Although not all the renewable energy sources are highly volatile, like wind power, a lot of unknowns will be thrown into the power grid. Constant power-supply predictions based on ever-changing weather (the wind may or may not blow at any given minute, and solar power goes down when clouds set in) will be vital to keep the power grid stable all the time.

Applying PI to more demanding domains

Smart grid is to make the power grid smarter. Our physical infrastructures consist of more than the just the power grid; we need, for example, gas, water, waste, transportation, government, street lights and traffic systems. Dave is working on the next topic beyond the power grid, which is the smart city. According to Dave, a smart city is defined differently by different people. But currently, US cities like Austin, Seattle, New York, and Chicago have their smart city projects. OSIsoft is involved in some of them, and a public announcement is coming shortly.

Collecting, aggregating, storing, and linking all sorts of data from its different sources would provide tremendous intelligence to a city. A utility at the conference reported that they collect 100,000 data per second. If we implement a system for a smart city, the number of data points would explode by the order of 2 to 3 magnitudes. That means millions of data per second would bombard the PI system. Even though the PI system is created to cope with a large amount of data of many kinds, at some point, they may have to alter their architecture and technologies to process such a massive amount of data. That makes me interested in talking to their technology visionary. Stay tuned for that in a coming blog.

When smart grid was first introduced, Cisco declared that the power grid would be much bigger than the Internet. From the data point of view alone, the amount of data produced and processed on the power grid is on a scale that none of us has experienced before. And with more-sophisticated monitoring technologies, the volume of data will even increase. The data collected may include equipment health, power flow, and quantity of power consumption. Simply collecting data does not do much good. We need to process what we collect—make heads and tails of it—to produce useful information for better operation and maintenance. This is the Big Data problem that is getting a lot of attention these days in ICT and other segments.

Usually, Big Data problems are due to the proliferation of SNSs, such as Facebook, Twitter, and LinkedIn. But with the advent of low-power and low-priced, yet very sophisticated, end devices and sensors, different kinds of Big Data problems are emerging, such as the one I just mentioned.

There are several companies that apply their software systems and tools to solve Big Data problems in a particular vertical market, such as the power industry. When I was covering data centers and their energy efficiency, I visited OSIsoft at its San Leandro, CA, headquarters in 2009. They collect data sent by end devices like sensors and their equivalents and store, analyze, and visualize the collected data to take appropriate actions for improving operations. Since that visit, my focus has expanded to include the power industry, which is only one of the markets OSIsoft addresses (see the other markets here).

Recently, I had an opportunity to attend their users conference in San Francisco.

I listened to several representatives of utilities and others in the power industry talk about their use of OSIsoft’s PI system. I also talked to Dave Roberts, Fellow and market Principal – Smart Cities, who is an expert in the power industry.

Dave Roberts

The following is my summary of our discussion, with my comments.

Some power grid basics

I am targeting this blog to very, very IT people and not to power people. So I think very simple, basic information is useful. The power grid is a big connected network of power lines. The power grid consists of two types of grids: transmission and distribution. Generated power is transmitted at a very high voltage via transmission lines to neighborhoods of consumers. Then the high voltage is transformed to much lower voltage, and power is delivered to consumers like you and me via the distribution grid. Because power must be consumed as it is produced, demand and supply need to be balanced all the time. Power on transmission lines is managed by each utility or by organizations called ISOs/RTOs (independent of utility companies) to make sure the balance of demand and supply is maintained—to keep the lights on. Also, as with computer networks, it is important to know the health and status of each device and all the equipment hanging from the grid. As in computer networks, such information is collected from multiple places in the grid. The number of collection points grows as more technologies are developed.

What OSIsoft does

Architecture

Although from my conversations with other OSIsoft people, I knew what business they were in, I just wanted to make sure who they are and what they do. They provide a software infrastructure system called PI to connect remote devices, gather/collect/aggregate data from them, and store and retrieve the collected data for further analysis, such as data analytics and visualization. They do not provide end devices like sensors or analytics engines. In other words, PI is one of the important components of the Internet of Things, M2M, or intelligent systems. Different people define the Internet of Things, M2M, and intelligent systems slightly differently, and the terms are often used interchangeably.

Here’s an oversimplified view of PI architecture.

My view on the conceptual view of PI architecture

PI is not an operating system but there is some analogy between PI and Windows. Windows provides a base operating environment for applications to run in. Microsoft in general does not provide any applications packages but provides this base plus some tools/utilities and libraries via APIs. Third parties exploit this platform to write applications. PI is similar and does not provide applications, including data analytics packages. So PI can be said to be a general platform and applications area agnostic.

This will continue to Part 2.

An ecosystem for intelligent systems consists of several players, such as chip, OS, middleware, end device, cloud service, back office processing and analytics providers, and system integrators. Ayla Networks, which is still in stealth mode, claims that they provide secure connectivity for an end-to-end solution for an intelligent system. They currently focus on the consumer market but do not rule out expansion into other areas.

I sat down with David Friedman, CEO of Ayla Networks, during the recent Design West to find out what they are up to.

David Friedman

Who they are

David was VP business development for a wireless chip company before. After selling it in 2010 , he saw a business opportunity. At that time, end devices were beginning to be connected to form the Internet of Things. But the ugly reality was that those thousands of end devices were very different from each other, with microcontrollers in a variety of architectures and operating systems, as compared with the nonembedded world dominated by Windows and Linux. All those differences sure were a hindrance to accelerating and proliferating the Internet of Things. David and his cofounders saw the need for a generic solution that could absorb these differences. That led to the formation of Ayla Networks.

David and his team started to work on his solutions. Using his background as a chip guy, he teamed up with STMicro because ST is a major player in the microcontroller market. Ayla Networks is a software company and does not deal with hardware, so this is a good combination. Chip vendors focus on how to design and develop new and better chips but are not experts in networking such as Berkeley sockets and SSL. In other words, companies should focus on their core competency and outsource the rest. In the same vein, application vendors are not experts in the lower layers of software that support applications. During Design West, I heard from several players that application vendors should outsource the lower layers and concentrate on their core business; that is, design and develop applications.

So David is saying “Come to us. We will absorb any protocols differences and security needs to support your applications. You do not need to worry about the lower layers and other infrastructure concerns.”

What they do

Ayla provides an end-to-end connectivity software; for example, to remotely control your AC from outside your home with your smartphone. If you implement something like that on your own, you need to develop lower-layer software for the smartphone, including secure interfaces with its OS and networking stack. Then you need to develop an application to work with that infrastructure. Then you need to worry about how to connect it to your target AC. Communication can be via cellular, WAN, LAN, or PAN. You need to choose the right one. Finally, on the target AC, some mechanism needs to be incorporated to receive data and control from your smartphone. For that, a small board with a communications chip on it must be inserted along with the lower-layer software. And as with your smartphone, you need to interface with that chip’s OS and networking stack, on top of developing applications.

What Ayla provides:

1. Client-side lower-layer software for applications
2. Networking solutions with security
3. Cloud services
4. Lower-layer software on target appliances

The client-side software can be integrated with your applications and downloaded from Apple Store and Google Play like other applications. Ayla provides whatever networking protocols are required by the applications. In addition, they provide cloud services to connect your client devices to target appliances. David did not elaborate on how they provide such services. Cloud services consist of cloud infrastructures and applications in the form of virtual machines. Because of the proliferation of inexpensive cloud infrastructures services, a startup like Ayla can afford to provide the cloud services. The lower-layer software on target appliances is the same as #1. Application developers can focus on their core business of developing applications without getting bogged down in lower-layer stuff.

Team

Now this seems to require a lot of technical expertise in several areas, such as embedded systems, networking, and cloud. Although these areas are closely related, no one person could address all of them. Although David did not reveal details about his team, he did say that he gathered technical people who had worked together for several years. People who like to innovate and have a passion to create something new are attracted to his team.

Devil is in the details

Many people have discussed controlling an AC from outside with a smartphone or a tablet, and that by itself is nothing new. David told me that now is the perfect time to bring their solutions to the market. Technologies have advanced and the market is opening up. An article by Reuters reports that by 2022 a typical household will own 50 Internet-connected devices, compared with 10 now. David said that we do not want 50 solutions for 50 devices but only a single solution so that any new device can easily belong to the existing network. He also emphasized that creating a supportable product is really, really difficult.

Their infrastructure pieces must be easy to:

• install
• configure with a lot of latitude
• scale
• implement with secure delivery

They claim that they have met all four requirements.

Big Data

They are in a perfect position to collect and aggregate data, but David did not reveal any future plan for business exploiting such a position. But he did not rule out the possibility, either. If I were an AC OEM, I would be very interested in analyzing control data sent by smartphones, to reflect on how to tune my AC features. David told me that the key to the use of Big Data is anonymization with the ability to opt in or out.

Energy consumption

What about power consumption? Smartphones eat a lot of power, and additional features like these would consume even more. David told me that his developers pay a lot of attention to curbing power consumption. Power-use optimization implemented as in the iPhone would attain energy efficiency.

We chatted a bit about power in general when everything is connected. My view was as follows:

1. Advantages: There are many advantages to deriving useful information from generated data that might be otherwise discarded. Some information can be used to save power.
2. Disadvantages: Unless we can intelligently select which data to collect, or keep, or discard, we will end up with a pile of useless data occupying a lot of storage and server equipment, wasting energy.

I think what David said about the disadvantages was interesting. He said that analyzing a vast amount of data, transforming them to a small number of useful data, and discarding the rest might do the trick. I do not know the doability of such a thing, but it is an interesting thought.

Future

David did not give me any concrete future plans, but this system can expand beyond the consumer segment to the commercial and industrial markets. I think there is a reasonable level of traction

in the consumer market at this point, and there will be greater demand later. In addition to a clear application for turning an AC on and off, I can think of a few more examples. Sprinklers for lawns are usually on timers, and occasionally they start to work even in the rain while you are not at home. Your remote device can override this. Or better yet, you can program sprinklers in conjunction with moisture-detecting sensors buried in the ground and with sensors for other local weather.

But I think the really big applications are in the commercial and industrial segments. I think it is very smart of Ayla to choose the consumer market first. There are two reasons. The first is that the commercial and industrial segments are known to be late adopters. The second is that if you target very specialized and sophisticated industry-grade equipment, how many people will know? But familiar appliances like ACs show up on many people’s radar screens; after success in the consumer market, Ayla can enter bigger markets.

The conversation stayed at a high level because they are still in a stealth mode, but a public announcement is forthcoming. Meanwhile, you can register to purchase their design kit by visiting here.

Fluid Operations (FluidOps) aggregates data from different sources and converts them with some intelligence for better analysis. I sat with Peter Haase, senior architect, and chatted about their Information Workbench, a comprehensive tool for collecting and analyzing data and visualizing useful information.

Peter Haase

Fluid Operations is located in Walldorf, Germany. SAP’s headquarters is there as well. They currently have no US office, but their website provides information in both German and English. Peter and other people from the company are fluent in English.

As in other areas, in the power business, utilities companies collect and aggregate various kinds of data in addition to meter-read data. They may monitor equipment on the distribution grid, such as transformers, switches, relays, and capacitor banks. The data from the equipment and the meter-read data may be generated at dramatically different speeds. In addition to dynamic and real-time data, some static data types like asset information, including equipment location, brand, model, specification, and service records, may be required to provide preventive maintenance and report malfunctions and failures. The FluidOps solution is to collect and aggregate data from multiple sources and then to translate each datum semantically to a common form so that it has more meaningful information associated with it. Since all the translated data are in the same form with more meaningful relationships among them, analytics becomes more effective and can lead to more appropriate action.

“Semantically” means that they convert collected data into their normal form, which is represented using the Resource Description Framework (RDF). I will not get into details here. Although it is not the same but in a way, it is similar to Entity-Relation model. An example diagram is shown here.

An example RDF graph (Source:Eric Miller)

All the data collected are converted into this format. The query language for RDF is SPARQL Protocol and RDF Query Language (SPARQL).

FluidOps Information Workbench consists of data integration and storage, data management, and presentation/interaction/UI customization layers. At the 30,000-foot view, it collects and associates data using semantic models from diverse industry segments. For example, the Linking Open Data Community project is an attempt to make data from different industry segments freely available, and for that, data are represented in RDF. The segments include media, geographic, publications, user-generated, governments, and life science. Their relationships are shown in the following diagram, which is maintained by Richard Cyganiak and Anja Jentzsch.

Linking Open Data cloud diagram, by Richard Cyganiak and Anja Jentzsch: available here. Data published in Linked Data format based on the Linking Open Data Community project.

Click each circle on the figure here (not the figure above) to drill down through each dataset.

The following figure illustrates how Information Workbench collects and associates data with other data to increase their value semantically.

The disparate sources include tweets, Facebook, YouTube, data.gov, office documents, and various video files.

The architecture of Information Workbench is shown below. It consists of a data integration and storage layer (green), data management (brown), and presentation, interaction, and UI customization (blue).

Fluid Operations looked at the availability of RDF datasets to exploit for effective analytics. Their current application areas include media and health care and life sciences. I asked Peter about its application to the power industry. He said they were not looking into that yet but may consider it if they get a research grant. I do not know whether a dataset is already available for the power industry, but I think it might help the industry to exploit something like this.

I talked about each utility’s operation, but if we look at each region, such as ISO/RTO, the regional power balance information and data are very useful. I would like to follow this as it grows.

I sat down with Metric Insights’ Marius Moscovici, CEO, and Steve Mock, COO, to find out what they were up to.

From left: Marius Moscovici, CEO, and Steve Mock, COO

When I interview companies at conferences, I usually want to know what they do in relation to the conference theme, and what their differentiation, future directions, and competition are. What they do seems to be fairly easy to understand, but differentiation and competition may not always be that easy to figure out. The NoSQL-related domain is still being defined as it moves in many directions at Big Data creation speed. Certainly, we need solid database technologies, including databases themselves and any utilities that enhance them, analytics engines, and good visualization tools.

For further expansion of this market, it is vital to get buy-in from the enterprise. William McKnight in his keynote speech advocated for putting NoSQL in the enterprise market and emphasized that only then would the NoSQL market become legitimate. Riding on what the enterprise already embraces would lower entry barriers for NoSQL-related technologies and services.

Business intelligence has been a big push in the enterprise market. Even before the age of Big Data, in a typical enterprise domain there was a large set of data not shared among different individuals and departments such as call centers, engineering, marketing, and HR. If a marketing campaign reflected call-center customer feedback, a company might be able to sell more of their products and services. Metric Insights’ goal is to enhance BI capability by increasing each datum’s value by associating it with more meaningful information. Because BI is already accepted in the enterprise segment, their goal is reasonable. They want to expand transitional BI into Big Data BI by sourcing data from Big Data as well as from traditional sources.

As shown below, Metric Insights collects data from multiple heterogeneous sources and adds context (relevant attributes, as shown in the figure) to make that data more effective and valuable. Metric Insights says this creates useful insights. The collected and context-enhanced data are stored in intermediate form (JSON) in a database. (By the way, some say that JSON will push XML out completely, and others say, not so fast, because the world is not built by Web alone. But that is not the focus of this blog.) When data have more attributes or context, you can provide more effective analytics because you have more relevant information on each datum.

Metric Insights’ system consists of data collection, augmenting data with other attributes (context), analysis, and visualization. By the way, I mentioned to those gentlemen that I was more interested in how the backend works than in their user interface (UI). Yes, the backend is important, but the front-end, the UI, is crucial in the BI segment. So gentlemen, I take back what I said. My comment came from my techie point of view. When you use a BI system, the first thing people pay attention to is the UI. Because not all BI users are data scientists, BI specialists, techies, geeks, or interested in how it works, its use should be easy and intuitive without lengthy training. When you present your BI tool, if it does not communicate its ease of use and simplicity, no one will pay any attention to it.

Metric Insights prepares typical dashboards for ease of use for a given application. The example below is for a sales database. Sales reps just select a pane to get to what they want instead of creating complex queries (like “what I would like to do”) to obtain the result they are looking for.

A typical screenshot of the UI for the sales database is given in the following. This example shows product releases and the number of daily sales demos made. When a new release is given, it is likely to have an increased number of daily demo requests. But if there is any sudden increase or decrease, you can take a look at that particular point and drill down because more relevant context is available and can be added as an annotation.

Architecturally, it uses a persistent cache to accommodate real and semi-real time data speed and store data in a local Mysql database as well as a document-based store (JSON format). Since it is document based, it is easy to add more information for each datum. Their system works with some well-known Big Data storage/databases and technologies, such as Cloudera, MongoDB, and Google BigQuery. Additionally, a secondary memory-based caching layer is used to optimize end-user access speed of analytics.

Their application areas include sales, production, inventory, and finance, and they are expanding their scope to include recruiting talent. This is an interesting area. It used to be difficult to gain information on each individual because publicly available personal information was very limited. A résumé is written to cast the best light on the job applicant, and references usually provide only positive comments. Now in the era of SNS, we can gather a vast amount of information on individuals when they are at their ease and off their guard, so to speak.

In a way, Metric Insights and Fluid Operations provide a similar product. They both collect data from multiple sources, convert them to a standard form with additional information, apply analytics, and visualize results. On the surface they are similar, but their focus and implementation differ significantly. Metric Insights uses context to enhance each datum and obtain insight, then stores it in JSON-based storage, which is more common for NoSQL players and more relaxed (and easier to manage) than the semantic model Virtual Operations uses.

I think both approaches are valid, and each has its good application areas. The market is still evolving and is big enough for both of them. I asked Metric Insights if they have considered the power industry as an application area. They have not considered it yet, but I think their product can be used for that. The power industry will face multiple Big Data problems, as they will have more real-time monitor data, such as meter-read, equipment status, data feeds from other systems like weather, static information like assets and service logs, and SNS. A utilities backoffice is filled with disjointed applications without much data sharing, which can be improved very much by something like this technology. I do not know how, but that is up to folks like Metric Insights and Fluid Operations.

Bob Wiederhold, President and Chief Executive Officer, Couchbase

Couchbase was formed about a year and a half ago (February 2011) by merging Membase (based in Mountain View, CA) and Couchone (based in Oakland, CA). Bob came from the Membase side, and the new Couchbase is located in Mountain View. Couchone was behind Apache CouchDB, which is open source written in Erlang (also open source) and with Apache License 2.0. Most of the original key developers and committers (including Damien Katz) for Apache CouchDB moved from Couchone to Couchbase. The original developers and committers still contribute to Apache CouchDB, but most efforts are now focused on Couchbase 2.0, which is a separate open source project also licensed with Apache License 2.0 and is being implemented mostly in C. This is because Erlang is a functional programming language and C is more appropriate to increase speed. I’ve dealt with many programming languages in the past but never touched Erlang before. Bob emphasized that while Couchbase is heavily influenced by Apache CouchDB, it is a completely separate open source project. Bob told me that the merger went very smoothly and they are now about 100 people strong.

+

||

I told Bob that I am confused by the NoSQL market, and he shared his view of it. It is interesting to hear different persons’ views on the market. Of course, there is not 100% agreement on the current market, but different views sometimes give me a pretty good perspective. He first distinguished the operational from the analytics engine, as below. The analytics engine is a Hadoop and its derivatives, such as Cloudera, Hortonworks, and MapR. Note that Couchbase is a partner of Cloudera.

Then he expanded the NoSQL area according to technology and placed NoSQL players in each category. I will not discuss each category in detail. Those who want more detail can reference here. Wikipedia classifies the NoSQL categories in a much finer way. For example, there are several subcategories for the key-value camp, and it distinguishes the graph-based from the object-based ones. By the way, at the conference I took a four-hour crash course given by Dan McCreary of Kelley-McCreary & Associates. It was a good tutorial, and if you had the chance, you could sit down and spend a half day in his class. I also thought a whitepaper by Couchbase, Navigating the Transition From Relational to NoSQL Database Technology, useful. It describes document-based technology in comparison with the relational database.

The current version of Couchbase (1.8) is in the key-value camp. But come the 2.0 release, it will become a document-based database completely. Each camp has it merits and shortcomings. Will one category dominate others and all its technologies be consolidated into one? As for what will happen to this market, Bob thinks the following.

He thinks the key-value and the document-based databases will be merged, and the merged area will be the biggest of the three new areas. The other two areas will not go away but remain somewhat a niche market. The document-based solution is powerful, as it can contain a document like an entire website as a blob (in a JSON format) and retrieve it. For this, JSON is becoming the de facto standard over XML; Couchbase also uses JSON. There are proponents for both JSON and XML. In the Web environment, JSON is far more suitable, but XML has its own areas of application. There are a few tools for converting JSON to XML and vice versa.

As for the competition, Bob was very frank in analyzing Couchbase against other players in the document camp, as in the following table. Checkmark size indicates how strong and complete an attribute is. Well, the size is somewhat arbitrary and just indicates relative competency. Bob said that Couchbase has put a lot of emphasis on performance, scalability, and always-on features (thus, big checkmarks) with less focus on ease of development (thus, a smaller checkmark). He also added that with the 2.0 release, ease of development will improve significantly since this is the point at which they become a document database. He said that his competition has put a lot of emphasis on ease of development but needs to work on other features.

Couchbase moves to focus on ease of development

Competition moves to other features

He said although ease of development requires a lot of expertise, other things, like performance, are very hard to improve. He told me Couchbase has a big advantage in that it can consistently provide sub millisecond latencies for reads and writes that is often 1/3 to 1/10 the latencies of other solutions. In addition, Couchbase can provide throughput/server that is often 2-4x higher than competing solutions (see http://bit.ly/NKJkVH and http://bit.ly/Qulb4R). The consistent low latency assures very responsive applications and the higher throughput per server means you need to buy less hardware and software than with other competing solutions.

The current application areas that use Couchbase include social gaming, ad and offer targeting, social networking, online business services, e-commerce, cloud data services, and mobile-to-cloud data synchronization. Because I am interested in the application of NoSQL technologies to power utilities companies for smart meters and monitoring (such as with sensors with SCADA access) many types and speeds of data (static, like asset data, to real-time meter-read data), I wondered how products like Couchbase can be applied. Bob’s view was that as the amount of sensor data and the frequency at which it is gathered increases, having a central database that can keep up with the inflow of data will become a challenge. NoSQL databases that have an ability to linearly scale up write throughput are an easy solution to capture the incoming data stream. Techniques like Couchbase Server’s incremental map reduce are ideal to provide real-time aggregation/analytics over the data.

I asked him about an ecosystem for each player. He thinks developing an ecosystem is vital for the success of Couchbase. The way things are, the market seems still very confused, but it is expanding rapidly.

First, for employers:

• You shouldn’t fire employees who are expressing lawful opinions that you do not agree with, even if you can. And based on where you are sited, or where your employees work, maybe you can’t. It’s a rotten world if every Republican CEO can fire Democrat employees, and vice versa.
• The NLRA needs to be examined in detail, and you should tailor social media policy documents to specifically state what the NLRA does and does not allow you to do, as an employer, and what the rights of workers are. Vague exhortations about professionalism are likely to be no help in a NLRB issue.

Stowe examines another post by Jon Hyman, Does your social media policy educate about being “profersonal?” who provides an anecdotal look at how folks, and their employers, might all get along, from a lawyer’s point-of-view.  Hyman provides some semi-sage advice from his legal idea of what folks ought to do to avoid having to deal with folks like him:

“Hyman then concludes with the most blatantly reactionary summing up imaginable:”

What’s the real lesson here? Social media is an evolving communication tool. Employees have not yet figured out what it means to be “profersonal.” Employees need to realize that anything they say online can impact their professional persona, and that every negative or offensive statement could lead to discipline or termination (even if employers can overreact in these situations). Until people fully understand that social media is erasing (has erased?) the line between the personal and the professional, these issues will continue to arise. It is our job as employers to help educate our employees about living in a “profersonal” world, even at the risk of offending the NLRB’s prickly sensitivities.

Prickly sensitivities aside, on the B2B side of things, we have quietly worked with many corporations to get their message out to their audience from both a top-down and bottom-up perspective.  NH has always believed that the message coming from both angles was important to our client’s customers, the media and those considering their technologies.  Since the vast majority of our clients provide a business-to-business range of products and services, no voice for the consumer market has been seriously considered.  Our biggest issue in doing this is building trust between the IT buyers and the folks within companies who speak the same language.

Harmonic convergence has not always been evident, but we’ve seen few occasions where bloggers have been threatened by job loss or political incursions into management’s druthers.  Since bottom-up speak has been viewed over the decades as dangerous by the legal icons inside every large technology company, vetting the columns and parsing the words has been excruciating at times.  For starters, lawyers and IT folks don’t share the same lexicon or world view on what customers, potential and real, need to know and when they need to know it.  Mix into that the marketing folks, whose territories have been invaded, and you often end up with a stalemate.  Nothing is worse to put all the time and money it takes to build the infrastructure, and end up with nothing.

Whether you’re in Texas or Hawaii, opinions often are not necessary when it comes to getting your message out across the social landscape.  Politics are never necessary in the social sphere any more than they are at trade shows and for that you can be sure.  Customers, new and old, really want to know how technologies like ERP, CRM and SAAS are going to affect their jobs, their lives and their bottom lines.  If publishing the facts about a disaster is looming on the company’s horizon, it’s always best to get out in front of it and address the solutions, so that users can see how you handle it.  You will find that being up front about problems may be your best campaign slogans and you will find that customers are your best advocate when the trust that you aren’t going to hide negatives.

…as with any end to a international school year there comes moments of reflection and looking back that generate ideas and new perspectives. This year has been a challenging one on many levels due to a host of odd circumstances and situations in my department and school that effected us all in the IT Department, and required the team to face up to serious challenges. We all pulled together, and hats of to the team for their outstanding capacity to deal with everything that came to us. This has had me thinking about something I like to call “organizational shrapnel” things that fly in your face unannounced from corners you would not expect things to come from. Sometimes I even refer to these as “drive by shootings” were suddenly you are confronted with a situation unexpected and most often you are not prepared to deal with, but react and deal with in the best of your own ability.

Organizational Shrapnel is a state which generates a lot of inefficiencies, frustrations, errors, in-competencies, and slowdowns in accomplishing tasks in an organization. It is the event, moment, action or process where people look around with some confusion and try to point fingers and there is no clear evidence or concrete event that you can associate to the inefficiencies, frustrations, errors, in-competencies, and slowdowns.

Why should this be an issue? I believe that organizational shrapnel and the level of it in an organization can make or break a good team, group or organization. I believe that when groups/organization engage in a task or project, for the objective to be met you need to be able to trust and know that the different players are all going to accomplish their respective tasks, and that the framework of resources supports accomplishing the task, project or action.

	- – - – - – - – - – - – - – - – - – - – - – - – - – - – - – - – - – - – - – - – - – - – - – - – - – - -
		Global Records Compliance: What You Need to Know
	- – - – - – - – - – - – - – - – - – - – - – - – - – - – - – - – - – - – - – - – - – - – - – - – - – - -

This process of teams collaborating and accomplishing a task together in any institution is complicated and quite complex, we all experience this and it is no surprise to most of us. I feel if organizations, and teams can better understand the causes of  organizational shrapnel and address the road blocks this tends to set  the stage for a more effective and smooth collaboration of a groups with in any organization.

So what are examples of organizational shrapnel? These are broad in scope, and come in all shapes and shades.

• A team works together and defines group agreements, but a member continually does not abide by these.
• A decision is made and agreed upon, but there is no clear implementation plan, or process for feedback and evaluation.
• A process or procedure is implemented within the organization, some stakeholder decide to follow it, others decide to not use it and use an alternative in isolation of the rest of the organization.
• A collaborative group works together, but all the actions and follow through are continually done by a few, and others play lip service to the work but are not participating actively and tend to block movement in the meeting setting
• An event, or task is organized and one of the stakeholders makes an error but does not report it, or share this out so others can support in correcting this.
• Not following through on things
• Looking at a situation within one perspective, and expecting others to follow through

…and many more, generally the pattern is that organizational shrapnel are small things which fall through, not completed tasks/steps, ignored agreements, tasks done inadequately, or where there is a lack motivation, drive by the players involved in completing the collaboration or task.

The frustration if organizational shrapnel becomes a habit and people accommodate to it, it can quickly become part of the organization’s culture and norms. The reality is all organization have a level of this, and the challenge is what environments and understandings can be shared to decrease it’s presence in any group or organization.

Organization, groups and teams who have a high capacity to understand and concretely deal with their organizational shrapnel create an environment where there is a greater motivation by the stakeholders of the organization which results in a sharp increase in creative flow generating innovation in moving forward.

John@http://beyonddigital.org/

So it was timely to listen to Brad Williams, Vice President, Utilities Product Strategy, who happened to be a keynote speaker at the recent Networked Grid 2011, to find out what Oracle is up to regarding smart grid.

Brad Williams

He covered quite a number of topics, listed below, but emphasized the importance of information and how to use it for utilities. The following is a summary of his talk.

These were his seven topics:

1. Smart grid investments

2. Consumer transactions

3. Electric vehicles

4. Intermittent renewable and electricity storage

5. Aging assets

6. Smart grid device management

7. Business intelligence and analytics

	- – - – - – - – - – - – - – - – - – - – - – - – - – - – - – - – - – - – - – - – - – - – - – - – - – - -
		Increase Your Revenue by Optimizing your Data Center
	- – - – - – - – - – - – - – - – - – - – - – - – - – - – - – - – - – - – - – - – - – - – - – - – - – - -

Smart grid investments:

• Investments continue but regulators require the added value of smart grid.
• Consumers need to be convinced of smart grid’s value.
• New entrants to the market include Google and Microsoft.
• Some negative media hype exists because of increased power bills.

Brad cited statistics on the growth in the amount of data collected as more smart grid applications are brought in to the utilities to run. The amount of data increases exponentially and may hit more than 800 TB (1 TB is 1,000 times 1 GB). Utilities will face this staggering amount of data and process it to obtain useful information.

Amount of data increases as new applications are added to a Tier-1 utility.

It may be hard to read the picture. These are the labels for the points shown on the graph:

• Advanced Distribution Automation
• Workforce Management Project
• Substation Automation System
• Mobile Data Goes Live
• Remote Terminal Unit (RTU) Upgrade
• GIS System Deployment
• OMS (outage management system) – 200-TB point
• Distribution Management Rollout
• AMI Deployment
• Programmable Communicating Thermostats (PCT) Come On-line – 700-TB point
• New devices in the home enabled by the smart meter – 800-TB point

Consumer transactions:

• Transactions are much more real-time, and IT needs to keep up with them.
• Utilities are educating consumers with key messages like:
  • Rate change and the benefits that come with smart grid
  • More-reliable services
  • Win-win for both utilities and consumers

Oracle’s take is that the discussions with consumers need to expand.

EVs: If managed well, they could be a killer app, but if not, they could disrupt the grid. Seventy-two percent of large utilities are evaluating EV adoption. Oracle thinks smart charging is the area where it can apply its expertise on management information to optimize the charging mechanism.

Intermittent renewable and electricity storage: Oracle is working with its customers to model, monitor, and manage renewable energy sources.

Aging assets: With smart grid, more information is available for devices and components and that will prolong their lives and make it possible to provide timely and proper maintenance.

Smart grid device management: Smart grid brings IT and OT (operations technology) together. An IT company like Oracle could provide services beyond traditional technology fields to the power market.

Business intelligence and analytics: As more data is collected, knowing how to make useful information out of it would give a company a competitive edge.

As you can see, IT companies can make the best of what’s going on in the smart grid market and enter the power utility area with their information technology and data analysis expertise.

Many International Schools have so many of their day to day tasks/work flows tied to IT systems that the non stop functioning becomes a non-negotiable. This topic came back in many of the conversations I had with fellow IT Directors from the ECIS region attending the conference. One thing which is becoming quite clear to me as I have these conversations IT Directors and IT Staff are being stretched more and more as new systems become a non negotiable critical part of the school day. With this a growing cultural expectations of the users and school of  access: anywhere, anytime. There is a developing cost to this for IT Directors and their teams. One is that you start juggling more and more tasks, your team which in many schools tend to be quite small, has to be able to deal with a wider variety of complex issues and integrated systems. A common case especially here in European International Schools, as systems get added, new programs or hardware, no extra people are brought in. So the task lists gets longer, the job description for many of the Technical Support team changes by the minute and somehow extra resources in human and money tend to be elusive. This too often not by fault but by necessity were International Schools work with small budgets and have often little flexibility to add people. There is a danger that can develop that you start having over stretched IT Departments providing 24/7 services but no organizational structure to support this growth, and then all your eggs are in one basket, hoping the IT Department small as it maybe can sustain and support the pressures and demands long term. Is there a breaking point? Is it sustainable?

	- – - – - – - – - – - – - – - – - – - – - – - – - – - – - – - – - – - – - – - – - – - – - – - – - – - -
		Increase Your Revenue by Optimizing your Data Center
	- – - – - – - – - – - – - – - – - – - – - – - – - – - – - – - – - – - – - – - – - – - – - – - – - – - -

I have no clear answer but what I am realizing and in conversations with others, IT Directors are starting to feel the stretch and strain. This comes in a mix of pressures that I personally feel has a cost to the health and well being of the person. As new systems get added, expectations become greater, connectivity and seamless availability of services 24/7 all add up to an intense mix of tasks and workload to sustain. This then becomes the responsibility of the IT Teams and the task of IT Director’s leadership to manage and facilitate these pressures. The IT Director who has to provide guidance, rally the tech. support folks (often under paid and under valued), creatively deliver solutions with tight budgets, and juggle the emotions, personalities and tensions often associated with the change process of integrating or introducing of new systems. procedures or hardware to different stakeholders.

The solution? Each international school has such unique dynamics that I do not think there is one simple solution and answer. The start is maybe having honest and candid conversations with the schools leadership teams and clearly articulating the expectations of services and up keep of systems that support the school day. Thinking strategically what support systems can be developed to ensure if new systems, hardware and 24/7 connectivity and delivery of services are expected how this plays out with your current set up and staffing. Looking for creative solutions to shift services to the cloud, or put more responsibility on the users to independently manage the devices and services they use to support their work day. This of course then becomes an important conversations regarding what professional development support will be provided, expectations of skills and managing a significant cultural expectation of who is responsible for what.

At some point the IT Team and IT Director need to also unplug and regroup, which for many of us is a challenge and near to impossible. Even when we are off campus or away the systems have to be managed, maintained and serviced, and we need to be connected to the various stakeholder groups we support, there is this growing expectation.

…as with any challenge engaging in a conversation, defining the expectations explicitly to all, and being willing to think beyond our own walls and perspectives can be the first step. This then tied to a long term strategic understanding that however essential and critical our schools services are, connected to this is a group of people trying to juggle a more and more complex set of dynamics and expectations.  We need to engage in an awareness that over stretching folks can have a negative impact on sustaining your own systems and anywhere anytime connectivity. I believe there is a potential for a better balance for all.

Let us have this conversation…………..

John@http://beyonddigital.org