Data Warehouse Strategy Considerations





Few of us would doubt that a data warehouse strategy should include the hardware/software platforms that are part of the architecture. These platforms can include multi-subject data warehouses, subject-specific data marts, master data management repositories, data warehouse appliances, enterprise information integration (EII) federated databases, operational data stores, and direct access to data in operational systems. However, when you're creating your data warehouse strategy, there are many other factors to consider as well.

Cloud Verses On-Premise Hosting

An organization's data warehousing platforms can be hosted on-premise or in the cloud. This is not an either/or decision and it is possible, and in many cases desirable, to embrace both. For example, "one-time" special analysis needs or remote field locations can use cloud-based platforms while headquarters might be supported by on-premise platforms.



Open Source Verses Proprietary Software

The choice between open source and proprietary commercial software is also not an either/or decision, and most organizations are likely to deploy both. Major decision criteria include functional requirements and vendor (or community) support as well as total cost of ownership.



Data Delivery Vehicles

Although one of the early goals in the design of a data warehouse architecture was to ensure the delivery of the right information to the right people at the right time, it should now be extended to include "regardless of where the recipient happens to be." We live in a mobile society and the ability to run an analysis and access information should not be limited to desktop PCs or laptops (or, for that matter, printed reports!).

Many organizations have standards relating to approved smartphones, tablets, and laptop computers, but most users do not wish to carry two devices -- one for personal use and one for work. Some users will try to connect their own personal devices to their organization's network, and you must educate them on the possible risks. Better yet, accommodate their needs with appropriate solutions.

In general, an organization needs to protect its data (and the privacy of its constituents). Access from mobile devices that can be easily lost or stolen makes that task more difficult. Techniques such as encryption and data wiping (after a specified number of password failures or when a device is reported missing) must be a part of the design, not a feature rushed into place following a major (and frequently costly) data breach. With that in mind, organizations might consider providing -- or subsidizing the cost of -- user-owned mobile devices under the condition that the organization can install appropriate software to remotely disable the device in the event it is lost or stolen.



Data Sources and Data Consistency

One of the major advantages of data warehousing is the ability to extract data from multiple heterogeneous operational systems and third-party data providers and transform it to match organizational data standards including formats, value lists, and units of measure when loaded into the data warehouse. Furthermore, it is desirable to load data into an enterprise data warehouse first and then distribute subsets to other data warehouse platforms. However, complete consistency is often unlikely because the data warehouse environment may include independent data mart platforms and/or allow analyses directly against operational systems whose data elements don't conform to organization standards that were established long after the application was initially deployed.


It is, therefore, extremely important that users understand the limitations of the data on individual data warehouse platforms. At a minimum, use a metadata repository (a master metadata manager?) that describes what data resides where along with any associated limitations. Furthermore, the same queries run against two platforms should produce consistent results, although they might have dramatically different response times. For example, if the data under analysis resides in both an enterprise data warehouse and a data warehouse appliance, the results should be the same (although the appliance might generate them faster).



Unstructured Data

With the growing recognition of the value of data contained in social media sources, an organization's data warehouse strategy should include the ability to accommodate unstructured data.



Data Security

Some data is simply too sensitive or too private to allow unrestricted access to everyone in your organization. For example, someone analyzing salary trends might only be allowed to see yearly average salary values by job code rather than individual salaries. Delivery of the right information to the right people does not imply delivery to everyone. Protective measures should be established to ensure that access is restricted as appropriate.

Although many factors influence the creation of an organization's data warehouse strategy, one of the most important characteristics of a good data warehouse strategy is its ability to respond to the changing needs of the organization. Make sure your data warehousing strategy is flexible enough to accommodate changes.

 

10 Mistakes to Avoid in a Business Intelligence Delivery (Part 2/ 5)

Mistake #3: Assuming service provider companies own everything about the successful delivery of the project.

This is yet another critical factor for a successful BI delivery. A service provider who has signed a contract to put the BI project into production definitely has ownership on the delivery. That being said, the delivery cannot be a success without active participation from the client and end users having in each stage and phase of the entire lifecycle. Service providers are specialized consultants who can give you options and best practices, much like a professional home decorator consultant. Because it's your home, you will have to give the consultant your input and exact specifications. If this does not happen, then the decorator will decorate the home according to his assumptions of your likes and dislikes, which you may or may not approve of. And if this happens at the last minute, then not only will you end up paying for the work that has already been done, but you will also invest more time and money on rework. Without active and adequate client involvement at every phase, no BI delivery can ever have an assured success.

 
 

Mistake #4: Bringing in a solution architect halfway into the project and assuming that he/she is going to magically fulfill all the deficiencies.

This is the most common scenario one sees in most of the BI projects. When things are not happening the way they should, the management thinks the immediate remedy is to get a solution architect. What one has to understand is that a solution architect cannot just walk in and wield a magic wand to set things right. The  solution architect s experience will determine how soon he/she can start delivering the value-adds. Also, the time at which you bring in the solution architect is a driving factor for success. Often when it comes to BI architecture, business users are from Mars and IT people are from Venus. To get them to a common platform is in itself a premium skill for a solution architect.

Every BI delivery must have a solution architect with expertise and wide skills in DW and BI. This is vital, as they bring a wealth of knowledge from reference architectures and similar implementations with them. This ready-recon eventually reduces the cost and time to implement technology solutions.

The best time to start their involvement is from the analysis stage itself. If not then, do it at least before you spend a large amount of time exploring technology options and assessing the appropriate solutions. Carefully set expectations of the value a solution architect would deliver. If you decide to engage a solution architect when your data model is near completion and expect the architect to do magic to make a performance-tuned and efficient design, it might be overexpecting, as things would have already crossed certain stages involving a good amount of time and cost. At that stage, again the issue resolution becomes more political than technical.

Trio of New Year's resolutions for Business Intelligence

Action Item #1: Power to the PeopleMore than half a decade ago, Web services and Web 2.0 gave shops a means to more easily expose analytic functionality to end users. What was lacking was a killer context. Thanks to social collaboration -- with its emphases on rich interactivity, information analysis, and give-and-take -- organizations can now conceive of at least one such context. BI vendors are even starting to build social capabilities into their tools, chiefly as a means to complement or enrich the analytic experience.

Pushing analytics out to rank-and-file users isn't something that could -- or should -- happen overnight, but it can't hurt to start talking about when and how to do so.

Action Item #2: Measure BI Impact
When it comes to analytics, a surprising number of shops haven't the foggiest idea what they're doing -- or how well they're doing it. In a survey published last year, for example, market-watcher IDC found that one-third of organizations lack the means to adequately assess the efficacy of their analytic investments -- in spite of the fact that BI itself is designed to help shops monitor and understand performance.


Which begs the question: what's more natural than using a BI tool to report on or assess the efficacy of its own adoption?

For this very reason, many BI vendors (including IBI, IBM, MicroStrategy Inc., Oracle Corp., SAP AG, and SAS Institute Inc.) include canned reports or analytics designed to help shops understand how their BI solutions are being used. The rub, then and now, is getting people to do so.

Organizations have lacked insight into the efficacy of their BI and data warehousing (DW) investments for far too long. It's a recurrent theme in market research from IDC, BIScorecard.com, and other sources. In most cases, the means to gain that insight is afforded by BI tools themselves.


It's time to get serious about this problem.


Action Item #3: Go Mobile


Last year, just before Christmas, the president and CEO of a scholarship and tuition management firm based in the Southeast gave her entire executive team iPads. She wasn't thinking particularly about BI, she concedes; she just sees the sleek iPad as a sensible complement to the bulky laptop computer. She's also excited about serving up dashboards, charts, and reports to iPad-toting executives -- perhaps someday to rank-and-file knowledge workers (see Action Item #1) -- using her company's SharePoint portal resources.

The iPad by itself isn't the answer to your mobile prayers, of course, but a mobile device will almost certainly figure in your BI future -- either as a complement to or replacement for the traditional (desktop) or transitional (laptop) PC.

Smartphones have been commonplace in corporations for almost a decade now. Samsung, Research in Motion Inc., and other vendors are unveiling iPad-like devices of their own. By 2012, IDC says, the combined base of smartphones and tablet computers will surpass that of the PC.

It's time to start thinking beyond the traditional PC. Some of the pieces -- such as smartphones or conventional tablet computers -- are already in place. What's missing is a top-down imprimatur. So why not make 2011 the Year of Mobile BI?


Analytics aren't just for analysts. Rank-and-file users are ready, willing, and -- most importantly -- able to assume more analytic responsibility.

In many cases, they're already interacting in analytic contexts, be it via blogging, tweeting, of participation in online forums. True, none of these activities is explicitly analytic; all the same, they provide a context in which people can test claims, gain insights, and make decisions. While they may never become analytic rock stars, they'll almost certainly benefit from being exposed to analytic functionality in their jobs.

10 Mistakes to Avoid in a Business Intelligence Delivery (Part 1/ 5)

Behind every success or failure are people. People are the only differentiators. Every data warehousing (DW) and business intelligence (BI) project, whether successful or not, teaches us something. It is generally on failures that we base our new success. Having said that, it s not always necessary that you fail to learn; you can also learn from others failures, 10 of which are discussed here.


Mistake #1: A non-BI background project manager managing the end-to-end delivery of a BI initiative.

BI project management requires different techniques and methods to succeed. The breakthrough in work process and methodology that form the foundation of data warehouse delivery include such concepts as iterations and phased delivery, and from a non-data warehouse perspective it's hard to appreciate how truly revolutionary and critical these concepts are for successful BI delivery.

A project manager who drives the complete BI initiative from end-to-end has to at least be educated  on the basics of DW and BI to be able to deliver the BI project successfully. No matter how successful  an individual maybe or how much expertise he/she has in managing non-data warehousing projects, he/she will never be able to deliver the DW projects successfully if he/she does not understand the phased delivery  approach of data warehouse. Most often it becomes very challenging to convince a non-DW project manager that the analysis and design phases in DW projects go side by side and not one after the other like in traditional project delivery. If this important aspect is ignored, then the schedule and budget are going to get hit, as one always encounters changing requirements in DW projects, whether he/she likes it or not. Additionally, the fallout would be arguments and politics rather than focusing on technical solutions.

Every project delivery requires a methodology, which a project manager uses to deliver the project successfully. A project manager who by definition plans, controls and reviews all project activities must understand that a data warehousing project delivery cannot use the traditional waterfall methodology. The data warehouse methodology must take into account the fact that the delivery of BI projects happens in iterations. The success of data warehousing projects is in its phased approach.

A project manager who is not knowledgeable about BI is not able to make appropriate staffing selections for his team. The team also suffers due to lack of guidance from the leadership role as much as the goal of the BI initiative would suffer because of the management.


Mistake #2: Being in a pleasing mode with the clients rather than concentrating on feasibility and value-add from the BI project.

The client sponsoring the DW project and end users have to accept the solution which is being built by the implementation team; there is no doubt about this fact. At the end of the day, the solution being built has to be liked and should demonstrate value-add to the clients. The time and effort spent on a particular initiative should demonstrate value for money. But a word of caution. In this process, the implementation team, which is most often a service provider company offering offshore support as well, should not get into the pleasing mode with the clients and users. It might not be practically possible to implement the client s entire wish list. This should be communicated in a strong but polite way. The requirements driving the DW initiative should be validated very critically so that the best solution can be built. What cannot be done should be communicated as clearly as you communicate what can be done. Clients will definitely appreciate and welcome this kind of assessment in the initial stages of a project rather than giving explanations on architecture and infrastructure just before production or in use or acceptance testing when it s too late.