Data Warehouse Strategy Considerations





Few of us would doubt that a data warehouse strategy should include the hardware/software platforms that are part of the architecture. These platforms can include multi-subject data warehouses, subject-specific data marts, master data management repositories, data warehouse appliances, enterprise information integration (EII) federated databases, operational data stores, and direct access to data in operational systems. However, when you're creating your data warehouse strategy, there are many other factors to consider as well.

Cloud Verses On-Premise Hosting

An organization's data warehousing platforms can be hosted on-premise or in the cloud. This is not an either/or decision and it is possible, and in many cases desirable, to embrace both. For example, "one-time" special analysis needs or remote field locations can use cloud-based platforms while headquarters might be supported by on-premise platforms.



Open Source Verses Proprietary Software

The choice between open source and proprietary commercial software is also not an either/or decision, and most organizations are likely to deploy both. Major decision criteria include functional requirements and vendor (or community) support as well as total cost of ownership.



Data Delivery Vehicles

Although one of the early goals in the design of a data warehouse architecture was to ensure the delivery of the right information to the right people at the right time, it should now be extended to include "regardless of where the recipient happens to be." We live in a mobile society and the ability to run an analysis and access information should not be limited to desktop PCs or laptops (or, for that matter, printed reports!).

Many organizations have standards relating to approved smartphones, tablets, and laptop computers, but most users do not wish to carry two devices -- one for personal use and one for work. Some users will try to connect their own personal devices to their organization's network, and you must educate them on the possible risks. Better yet, accommodate their needs with appropriate solutions.

In general, an organization needs to protect its data (and the privacy of its constituents). Access from mobile devices that can be easily lost or stolen makes that task more difficult. Techniques such as encryption and data wiping (after a specified number of password failures or when a device is reported missing) must be a part of the design, not a feature rushed into place following a major (and frequently costly) data breach. With that in mind, organizations might consider providing -- or subsidizing the cost of -- user-owned mobile devices under the condition that the organization can install appropriate software to remotely disable the device in the event it is lost or stolen.



Data Sources and Data Consistency

One of the major advantages of data warehousing is the ability to extract data from multiple heterogeneous operational systems and third-party data providers and transform it to match organizational data standards including formats, value lists, and units of measure when loaded into the data warehouse. Furthermore, it is desirable to load data into an enterprise data warehouse first and then distribute subsets to other data warehouse platforms. However, complete consistency is often unlikely because the data warehouse environment may include independent data mart platforms and/or allow analyses directly against operational systems whose data elements don't conform to organization standards that were established long after the application was initially deployed.


It is, therefore, extremely important that users understand the limitations of the data on individual data warehouse platforms. At a minimum, use a metadata repository (a master metadata manager?) that describes what data resides where along with any associated limitations. Furthermore, the same queries run against two platforms should produce consistent results, although they might have dramatically different response times. For example, if the data under analysis resides in both an enterprise data warehouse and a data warehouse appliance, the results should be the same (although the appliance might generate them faster).



Unstructured Data

With the growing recognition of the value of data contained in social media sources, an organization's data warehouse strategy should include the ability to accommodate unstructured data.



Data Security

Some data is simply too sensitive or too private to allow unrestricted access to everyone in your organization. For example, someone analyzing salary trends might only be allowed to see yearly average salary values by job code rather than individual salaries. Delivery of the right information to the right people does not imply delivery to everyone. Protective measures should be established to ensure that access is restricted as appropriate.

Although many factors influence the creation of an organization's data warehouse strategy, one of the most important characteristics of a good data warehouse strategy is its ability to respond to the changing needs of the organization. Make sure your data warehousing strategy is flexible enough to accommodate changes.

 

10 Mistakes to Avoid in a Business Intelligence Delivery (Part 2/ 5)

Mistake #3: Assuming service provider companies own everything about the successful delivery of the project.

This is yet another critical factor for a successful BI delivery. A service provider who has signed a contract to put the BI project into production definitely has ownership on the delivery. That being said, the delivery cannot be a success without active participation from the client and end users having in each stage and phase of the entire lifecycle. Service providers are specialized consultants who can give you options and best practices, much like a professional home decorator consultant. Because it's your home, you will have to give the consultant your input and exact specifications. If this does not happen, then the decorator will decorate the home according to his assumptions of your likes and dislikes, which you may or may not approve of. And if this happens at the last minute, then not only will you end up paying for the work that has already been done, but you will also invest more time and money on rework. Without active and adequate client involvement at every phase, no BI delivery can ever have an assured success.

 
 

Mistake #4: Bringing in a solution architect halfway into the project and assuming that he/she is going to magically fulfill all the deficiencies.

This is the most common scenario one sees in most of the BI projects. When things are not happening the way they should, the management thinks the immediate remedy is to get a solution architect. What one has to understand is that a solution architect cannot just walk in and wield a magic wand to set things right. The  solution architect s experience will determine how soon he/she can start delivering the value-adds. Also, the time at which you bring in the solution architect is a driving factor for success. Often when it comes to BI architecture, business users are from Mars and IT people are from Venus. To get them to a common platform is in itself a premium skill for a solution architect.

Every BI delivery must have a solution architect with expertise and wide skills in DW and BI. This is vital, as they bring a wealth of knowledge from reference architectures and similar implementations with them. This ready-recon eventually reduces the cost and time to implement technology solutions.

The best time to start their involvement is from the analysis stage itself. If not then, do it at least before you spend a large amount of time exploring technology options and assessing the appropriate solutions. Carefully set expectations of the value a solution architect would deliver. If you decide to engage a solution architect when your data model is near completion and expect the architect to do magic to make a performance-tuned and efficient design, it might be overexpecting, as things would have already crossed certain stages involving a good amount of time and cost. At that stage, again the issue resolution becomes more political than technical.

Trio of New Year's resolutions for Business Intelligence

Action Item #1: Power to the PeopleMore than half a decade ago, Web services and Web 2.0 gave shops a means to more easily expose analytic functionality to end users. What was lacking was a killer context. Thanks to social collaboration -- with its emphases on rich interactivity, information analysis, and give-and-take -- organizations can now conceive of at least one such context. BI vendors are even starting to build social capabilities into their tools, chiefly as a means to complement or enrich the analytic experience.

Pushing analytics out to rank-and-file users isn't something that could -- or should -- happen overnight, but it can't hurt to start talking about when and how to do so.

Action Item #2: Measure BI Impact
When it comes to analytics, a surprising number of shops haven't the foggiest idea what they're doing -- or how well they're doing it. In a survey published last year, for example, market-watcher IDC found that one-third of organizations lack the means to adequately assess the efficacy of their analytic investments -- in spite of the fact that BI itself is designed to help shops monitor and understand performance.


Which begs the question: what's more natural than using a BI tool to report on or assess the efficacy of its own adoption?

For this very reason, many BI vendors (including IBI, IBM, MicroStrategy Inc., Oracle Corp., SAP AG, and SAS Institute Inc.) include canned reports or analytics designed to help shops understand how their BI solutions are being used. The rub, then and now, is getting people to do so.

Organizations have lacked insight into the efficacy of their BI and data warehousing (DW) investments for far too long. It's a recurrent theme in market research from IDC, BIScorecard.com, and other sources. In most cases, the means to gain that insight is afforded by BI tools themselves.


It's time to get serious about this problem.


Action Item #3: Go Mobile


Last year, just before Christmas, the president and CEO of a scholarship and tuition management firm based in the Southeast gave her entire executive team iPads. She wasn't thinking particularly about BI, she concedes; she just sees the sleek iPad as a sensible complement to the bulky laptop computer. She's also excited about serving up dashboards, charts, and reports to iPad-toting executives -- perhaps someday to rank-and-file knowledge workers (see Action Item #1) -- using her company's SharePoint portal resources.

The iPad by itself isn't the answer to your mobile prayers, of course, but a mobile device will almost certainly figure in your BI future -- either as a complement to or replacement for the traditional (desktop) or transitional (laptop) PC.

Smartphones have been commonplace in corporations for almost a decade now. Samsung, Research in Motion Inc., and other vendors are unveiling iPad-like devices of their own. By 2012, IDC says, the combined base of smartphones and tablet computers will surpass that of the PC.

It's time to start thinking beyond the traditional PC. Some of the pieces -- such as smartphones or conventional tablet computers -- are already in place. What's missing is a top-down imprimatur. So why not make 2011 the Year of Mobile BI?


Analytics aren't just for analysts. Rank-and-file users are ready, willing, and -- most importantly -- able to assume more analytic responsibility.

In many cases, they're already interacting in analytic contexts, be it via blogging, tweeting, of participation in online forums. True, none of these activities is explicitly analytic; all the same, they provide a context in which people can test claims, gain insights, and make decisions. While they may never become analytic rock stars, they'll almost certainly benefit from being exposed to analytic functionality in their jobs.

10 Mistakes to Avoid in a Business Intelligence Delivery (Part 1/ 5)

Behind every success or failure are people. People are the only differentiators. Every data warehousing (DW) and business intelligence (BI) project, whether successful or not, teaches us something. It is generally on failures that we base our new success. Having said that, it s not always necessary that you fail to learn; you can also learn from others failures, 10 of which are discussed here.


Mistake #1: A non-BI background project manager managing the end-to-end delivery of a BI initiative.

BI project management requires different techniques and methods to succeed. The breakthrough in work process and methodology that form the foundation of data warehouse delivery include such concepts as iterations and phased delivery, and from a non-data warehouse perspective it's hard to appreciate how truly revolutionary and critical these concepts are for successful BI delivery.

A project manager who drives the complete BI initiative from end-to-end has to at least be educated  on the basics of DW and BI to be able to deliver the BI project successfully. No matter how successful  an individual maybe or how much expertise he/she has in managing non-data warehousing projects, he/she will never be able to deliver the DW projects successfully if he/she does not understand the phased delivery  approach of data warehouse. Most often it becomes very challenging to convince a non-DW project manager that the analysis and design phases in DW projects go side by side and not one after the other like in traditional project delivery. If this important aspect is ignored, then the schedule and budget are going to get hit, as one always encounters changing requirements in DW projects, whether he/she likes it or not. Additionally, the fallout would be arguments and politics rather than focusing on technical solutions.

Every project delivery requires a methodology, which a project manager uses to deliver the project successfully. A project manager who by definition plans, controls and reviews all project activities must understand that a data warehousing project delivery cannot use the traditional waterfall methodology. The data warehouse methodology must take into account the fact that the delivery of BI projects happens in iterations. The success of data warehousing projects is in its phased approach.

A project manager who is not knowledgeable about BI is not able to make appropriate staffing selections for his team. The team also suffers due to lack of guidance from the leadership role as much as the goal of the BI initiative would suffer because of the management.


Mistake #2: Being in a pleasing mode with the clients rather than concentrating on feasibility and value-add from the BI project.

The client sponsoring the DW project and end users have to accept the solution which is being built by the implementation team; there is no doubt about this fact. At the end of the day, the solution being built has to be liked and should demonstrate value-add to the clients. The time and effort spent on a particular initiative should demonstrate value for money. But a word of caution. In this process, the implementation team, which is most often a service provider company offering offshore support as well, should not get into the pleasing mode with the clients and users. It might not be practically possible to implement the client s entire wish list. This should be communicated in a strong but polite way. The requirements driving the DW initiative should be validated very critically so that the best solution can be built. What cannot be done should be communicated as clearly as you communicate what can be done. Clients will definitely appreciate and welcome this kind of assessment in the initial stages of a project rather than giving explanations on architecture and infrastructure just before production or in use or acceptance testing when it s too late.

Capitulo 5. Diseño DWH - El Modelado Dimensional (FK-MODI)

En este capitulo veremos como se crea el documento denominado "Modelado Dimensional" dentro de la etapa 5 (Diseño y arquitectura del DWH). El desarrollo de este documento servir? para representar los requerimientos de los usuarios desde el punto de vista de un modelo dimensional y a su vez sirve como entrada para desarrollar el m?delo lógico de la base de datos (DM o DWH).


http://blip.tv/file/get/Falconeris-Capitulo5DiseoDWHElModeladoDimensionalFKMODI590.mp4

Five Simple Steps to Better Decisions

Business processes seem to come in two flavors: those that produce transactions or content and those that produce decisions. The quality of decisions from the latter category often drives the trajectory of the business. Well-executed, insightful decisions can lead to superior results.

1. Focus on Processes that Matter to Your Business

Organizations improving insightful decision-making carefully pick the key processes and operational variables on which to focus. Clear alignment exists between a successful organization's market strategy and its processes and operating metrics to implement the strategy. W. Chan Kim and Renee Mauborgne, in their groundbreaking book, "Blue Ocean Strategy," developed an interesting approach in which they recommend picking operational variables in the context of strategy development. They point to well-known examples of companies with clearly differentiated strategies, including Southwest Airlines and Cirque du Soleil.

In my own work, particularly in the high tech electronics industry, I have seen clients select variables that include forecast accuracy, order fulfillment rate and inventory levels. Making planning decisions on a weekly basis at the SKU level based on insights across those variables resulted in tremendous improvements in all three.

2. Stay Focused on Your End Goal

The improvements you target should be expressed as changes in the specific selected variables. For example, if the goal is to reduce inventory by 30 percent, the initiative should fit that objective clearly. This kind of Deming approach of "you get what you measure" is well-documented, but it is surprising how many organizations do the first step without then taking the time to set clear objectives in the second. Deriving insights from a business process requires a good balance of freedom to efficiently explore information and decision alternatives coupled with a clear idea of the objective.

3. Ensure Your Data Supports Your Insights

Taking into account the processes, variables and objectives selected in the first two steps, the third step in improving decisions is to determine the readiness of your data and infrastructure to support the kind of insights required. Organizations often get caught in the trap of believing that their data or infrastructure are not up to the task and assuming that progress cannot be made without solving those issues. And yet, decisions must still be made, and it falls on business analysts to cobble together information manually and come to meetings armed with spreadsheets. These discussions based on suspect data often lead to finger pointing and fact questioning instead of insightful decisions. My observation is that if the data is suitable to drive these required, but often ineffective, discussions, would it not make more sense to leverage the data in a smarter way to derive insights more systematically and in a way that improves over time?

One of these techniques is to provide analytic reports showing all variations of a particular data field along with their owners. The process designers indicate which field variant is authoritative for a particular value, and technology can be used to manage the communication with other owners as they align their data. As alignment is achieved, the quality of the insights steadily improves. This "peer pressure" approach to data cleansing at the source is reminiscent of rating systems used for sites like eBay. There is incentive to getting the information right at its sources because everyone sees the impacts of good and bad data downstream.

This technique is distinct from the traditional approach of creating large data warehouses that attempt to consolidate schemas and provide highly cleansed enterprise data from a central source for driving analyses and processes. Many organizations have struggled with the data warehouse approach, in part because their businesses don't remain static long enough to even finish the warehousing project. In my own work, I typically leverage warehouses that are in place, at whatever level of completion, and then use the peer pressure approach to fill in the gaps and address new gaps as they emerge.

4. Parlay Processes & Insights into Smarter Decisions

The fourth step is to design and engineer the process and business analytic capabilities required to produce the insights and execute the resulting decisions. This work might seem straightforward, but it is fraught with subtleties and traps typically resulting from experience biases among the team involved in the work. For example, an IT team charged with deploying a company's business intelligence technology of choice would naturally focus on the reports required. The reports are a critical part of deliverables, but if the business analysts still have to manually transform the information and engage in offline or disconnected interpretation discussions spanning a company's functions, driving insightful decisions remains difficult.

Alternatively, if the team is adept at business software development that supports transaction or content production processes, the tendency is to try to develop analytical processes using the same methodology. This typically results in elongated development cycles and solutions that still miss the mark. Improving insight requires a careful combination of flexibility and context management in some kind of guided analytics environment, as opposed to an exact, step-by-step approach.

If the team comes from a process development or consulting background, the two traps I see most are biasing the work more toward the process than the result and producing one-time deliverables that may not transition well into an ongoing change vehicle.

While many of the skills of these teams are often highly valuable, agile development processes coupled with the right amount of business-focused domain expertise are more suitable for business analytics. Getting capabilities in the hands of the process stakeholders quickly and then letting them evolve as the methods of gaining insights emerge usually adds more value quicker than locking down exact requirements and following traditional development methods. And it is equally important to ensure that the resulting process captures the entire insight loop, including planning, reporting, analysis, collaboration, decision-making and execution.

5. Use Your New Processes to Drive Improvements

The fifth step is to operate the new process and drive the targeted improvements. Here, it is important to make sure resources are provided for properly interacting with the process, data and stakeholders to facilitate the emergence of insights and decisions. Initially, the new process might require more work than the old process, especially until the stakeholders get comfortable with the differences in the new decisions versus what they would have done in the past. This initial increase in work should be planned for, and if your program is successful you should soon see a much sharper net decrease in work versus the old process. The best insight-driven processes eventually require tremendous effort to stop, as opposed to tremendous efforts to keep them running.

This five-step approach blends a number of disciplines, and most organizations will not have all of the skills or required technologies readily available. You don't have to go it alone or wait to get started until your team is fully in place, though. Business models and other resources are emerging quickly to help organizations holistically with these kinds of programs and will be of tremendous value as you develop your program. As you move through the steps, you can define the gap between the resources you have and those that are needed and build your case around the target variable improvements that process insights will incrementally deliver to your business.

SaaS BI Tools: Better Decision Making for the Rest of Us

Simple business decisions, each of which impacts a company's performance and efficiency, are made every day, at every level of an organization, by workers in every department. But conventional business intelligence (BI) tools are often not available to most decision makers and are typically designed for use only by trained business analysts. Software as a service (SaaS)–based BI tools are designed to help the millions of people in non-IT "lines of business" (LOBs) who struggle every day with the task of mining Microsoft Excel spreadsheets and other unstructured data sources when performing everyday tasks such as making sales forecasts, planning for resource utilization, or servicing customer accounts. Especially in this time of limited budgets and uncertain futures, inexpensive, easy-to-deploy SaaS BI can help companies put easy-to-use data mining and reporting tools for smart decision making into the hands of more employees and uncover the real "geniuses" of decision making hidden in every department.


Benefits of SaaS BI

BI offerings delivered via the cloud provide tremendous additional benefits of scale and efficiency,lower cost, and better consumption of cloud and local data sources, and they are changing the way businesses license, deploy, and utilize BI to support decisions at their companies. Some benefits of SaaS BI are as follows:


Access by more employees to more data. Key beneficiaries of the trend toward SaaS BI have been the millions of people in non-IT lines of business who struggle every day with the task of mining Excel spreadsheets and other unstructured data sources when performing everyday tasks such as making sales forecasts, planning for resource utilization, or servicing customer accounts. Users of LOB applications produce the production data that drives BI requirements, and the powerful BI reporting and analysis capabilities are especially impactful in the hands of the users who created the data, resulting in greater adoption and utilization. Every business can be more efficient by putting better reporting and analysis tools into the hands of the LOB and departmental employees who are the subject matter experts in their domains. SaaS BI can make their jobs easier by providing browser-based access to sophisticated but easy-to-use data mining and reporting tools and uncovering the "geniuses" of decision making hidden in every department.

Business optimization for hard times. SaaS-based analytics can help companies be more resourceful in volatile times by helping them identify cost savings, efficiencies, and opportunities for process improvement they may have otherwise "missed in the data."


Faster "time to value" for a quicker return on investment. Implementations of SaaS BI solutions can be far faster and less expensive than implementations of conventional solutions. Consider that building a traditional BI solution with a data warehouse implementation, data normalization, and data marts for data staging by query systems typically requires between 6 and18 months, sometimes longer. By contrast, SaaS BI deployments typically require 2 to 4 months, and SaaS vendors cannot book revenue until the implementation is complete — a situation in which both buyer and seller are equally incented to decrease what some vendors call "time to value."

Streamlined architecture, with zero infrastructure. Unlike on-premises BI systems, SaaSbased BI is hosted by a vendor. Users access the various modules (for example, analysis,reporting) securely via any Web browser. From a systems architecture standpoint, this method is optimal because it does not impose an ongoing computing burden on back-office production systems, and because the application is hosted by the SaaS provider, users do not need to maintain an onsite data warehouse. Users conduct their secure sessions via a Web browser, so there is no client software to install, and users are always assured of running the most recent, optimized version of the application code because SaaS applications are not "rolled out" like conventional applications; they are simply upgraded and optimized on an ongoing basis.



Ability to tap operating expense (opex) budgets versus capital expense (capex) budgets. Because SaaS solutions are licenses as subscriptions, their license cost is a monthly, predictable expense and does not require a one-time up-front payment for licenses as conventional software. Further, the ongoing support costs to run associated hardware, management, and integration tools and middleware and hire and train staff members to support on-premises applications are substantial, and nonmaintenance support costs are typically booked as capex. Because these budgets will be flat in 2010–2011, SaaS solutions give users a chance to get access to BI and analytics tools much faster, using opex funds that might reside in their LOB budgets.

Better alignment of business goals. Business units consuming IT resources sometimes feel discordance between the technology they know they need to have to produce good business outcomes and the tools their IT staff has the skills and bandwidth to deploy. But IT is typically a cost center, and its priorities don't always align with LOB requirements. SaaS-delivered BI helps business units get business done and helps align the goals of the business unit with its technology tools.