The new configurations include Oracle Big Data Appliance X3-2 Starter Rack, containing six Oracle Sun servers within a full-sized rack with redundant Infiniband switches and power distribution units; as well as Oracle Big Data Appliance X3-2 In-Rack Expansion, which includes a pack of six additional servers to expand the above configuration to 12 nodes and then to a full rack of 18 nodes.

Both new systems include the existing software stack for Oracle Big Data Appliance X3-2: Oracle Linux, Oracle Hotspot Java Virtual Machine, Cloudera’s Distribution Including Apache Hadoop (CDH), Cloudera Manager and Oracle NoSQL Database.

Integrating Hadoop into existing enterprises, Oracle also provides powerful SQL access to HDFS data. This enables organizations to leverage their existing Oracle SQL skillsets and tools to seamlessly query and analyze data stored in Hadoop.

Additionally, Oracle Big Data Appliance X3-2 (full rack configuration) is now available through Oracle Infrastructure as a Service (IaaS), where organizations can obtain Oracle Big Data Appliance X3-2 on-premise, behind their firewall, for a monthly fee. With Oracle IaaS for Oracle Big Data Appliance, organizations can now eliminate upfront capital expenditures for their Big Data needs.

The Oracle Big Data Appliance X3-2 product family is comprised of Engineered Systems that simplify the implementation and management of Big Data by integrating hardware and software to acquire, organize and analyze Big Data.

Supporting Quotes

“With these new additions to the Oracle Big Data Appliance product family, Oracle is meeting the needs of a wider number of customers and expanding our offering to handle everything from the first proof of concept to a large scale enterprise deployment,” said Çetin Özbütün vice president, Data Warehousing and Big Data Technologies, Oracle. “We are focused on bringing the power of big data to our existing enterprise customers, with an integrated platform of engineered systems and key capabilities like SQL support for HDFS.”

“Oracle and Cloudera share a commitment to making Hadoop more accessible to an ever-broader set of users and organizations,” said Tim Stevens, VP of Business and Corporate Development at Cloudera. “These new options for deploying Oracle Big Data Appliance are ideal for the many enterprises that are just beginning to implement Hadoop and need a solution that enables them to start small and scale as their requirements grow.”

Source: Oracle.

The BI Survey 13 is strictly vendor-independent. BARC does not accept vendor sponsorship of the Survey, and the results are analyzed, compared and published without any vendor involvement.

To take part in this year’s survey, visit:
https://digiumenterprise.com/answer?link=1348-T9K4SWA8

As a participant, you will:

• Receive a summary of the results from the full Business Intelligence survey;
• Be entered into a draw to win one of ten $50 Amazon vouchers;
• Ensure that your experiences are included in the final analyses.

BARC’s annual Business Intelligence survey gathers input from a large number of organizations to analyze their buying decisions, implementation cycles and the benefits they achieve from using Business Intelligence software.

You will be able to answer questions on your usage of a BI tool from any BI vendor. Your answers will be used anonymously and your personal details will never be passed on to Business Intelligence vendors or other third parties.

Business and technical users, as well as vendors and consultants, are all encouraged to participate.

The BI Survey 13 should take about 20 minutes to complete. For further information, please contact Nikolai Janoschek at BARC (njanoschek at barc.de).

Click on the image below to take part in The BI Survey 13

The newest release of Informatica’s Powercenter, version 9.5, is now in the survey and the ETL products SAS Data Integration Studio, SAP Data Services, Talend’s Open Studio for Data Integration, version 5.2 and CloverETL have undergone updates.

SAP’s Data Services product has become so easy to use that they have set the standard for other products to measure themselves against.

The two products that have been added are:

• Syncsort DMExpress 5.5;
• Expressor (QlikView) Data Integration Platform 3.7. ( now QlikView)

If you want to compare all the major ETL tools on 90 criteria, or a selection of them, you can purchase the ETL Tools & Data Integration Survey 2013.

The information and insights it contains will save you a lot of time and the survey is 100% vendor independent and free from any vendor bias.

• Oracle today also announced availability of a new release of Oracle Business Intelligence Foundation Suite that features 87 new product capabilities and enhancements including new visualizations, contextual interactions, performance and scale improvements, optimizations for Oracle Exalytics, and simplification for end users, developers and administrators.
• With Oracle Exalytics, organizations can deliver custom and packaged analytic and performance management applications that run faster, are easier to use, and support more users than ever before. The engineered system enables customers to complement their dashboard and reporting projects with scenario modeling, planning and forecasting all running in a single environment.
• To provide ultimate performance and scalability with reduced deployment complexity and lower Total Cost of Ownership, Oracle Exalytics is architected as an engineered system featuring the industry-leading Oracle BI Foundation Suite and Oracle TimesTen In-Memory Database for Exalytics enhanced for an Oracle server designed for in-memory analytics.
• With interactive real-time analysis enabled by Oracle Exalytics, organizations can maximize profitability, increase revenues and market share, and react more quickly to changing business conditions.

Breakthrough Innovations Deliver Optimized In-Memory Analytics

• A complete, high-performance engineered system, Oracle Exalytics features Oracle’s Sun Fire server with 1 Terabyte of RAM, powered by the Intel Xeon processor E7-4800 with a total of 40 processing cores, 40 Gb/s InfiniBand and 10 Gb/s Ethernet connectivity, and Integrated Lights Out Management.
• Oracle BI Foundation includes an extensive set of new capabilities in Oracle Business Intelligence Enterprise Edition and Oracle Essbase for parallel performance and in-memory analytics that deliver to users real-time speed-of-thought visual analysis with sub-second responsiveness.
• Oracle TimesTen In-Memory Database for Exalytics is a new release of the industry-leading in-memory RDBMS specifically designed for analytics.
  • With new support for OLAP grouping sets, built-in analytic functions, and columnar compression, Oracle TimesTen In-Memory Database for Exalytics delivers instantaneous response over large data sets.
• Oracle Exalytics simplifies management with an innovation called heuristic adaptive in-memory caching that calculates what data is best stored in-memory for overall best performance, and adapts to changing workloads. Unlike other appliances or in-memory tools, Oracle Exalytics does not constrain applications only to data that fits in-memory, but also leverages connected databases, data warehouses and OLAP-tools.
• Benchmarking and customer testing of Oracle Exalytics demonstrates relational OLAP (ROLAP) reporting and dashboard performance gains of 10x to 100x, and multidimensional OLAP (MOLAP) modeling performance increases of up to 79x.

Fastest Performance for Over 80 Oracle BI and EPM Applications

• Eighty-plus Oracle BI and Oracle Hyperion performance management applications are available today for organizations to use in conjunction with Oracle Exalytics without application changes. These include:
  • All horizontal and industry-focused Oracle BI Applications such as Oracle Financial Analytics, Oracle Human Resources Analytics, Oracle Sales Analytics, and Oracle Procurement and Spend Analytics, among others; and,
  • Oracle Hyperion Planning for planning and forecasting using more detailed data and scaling to support larger numbers of users with reduced plan cycle time and greater plan accuracy.
• Oracle BI Foundation Suite includes built-in mobile support for iPad and iPhone. As part of an Oracle Exalytics solution, on-the-go mobile analysis scales to support many thousands of concurrent users with interactive responsiveness.

Open System Supports Oracle and Third-Party Enterprise Data Sources

• An open solution, Oracle Exalytics is suited for use in heterogeneous IT environments, and can access and analyze data from Oracle or third-party relational, OLAP or other data sources including IBM DB2, IBM Netezza, Microsoft SQL Server and Analysis Services, SAP Business Information Warehouse (BW), Sybase Adaptive Server Enterprise (ASE), and Teradata Warehouse, among others, in any combination.
• Oracle Exalytics integrates with and complements Oracle Exadata Database Machine via 40 Gb/s InfiniBand connection and software enhancements to deliver unparalleled query performance and sub-second interactive visualization across very large data sets.

“One of the real advantages of Oracle Exalytics In-Memory Machine is of course the speed,” said John Anker Moeller, Senior Vice President, Nykredit. “Oracle Exalytics will give us around 35x to 70x faster reports.”

1. Marcel de Wit – See another discussion on LinkedIn (still active; Dutch)

2. Daan van Beek – Thanks Marcel, I did read the comments of that discussion too, it was the reason I started this discussion actually. After reading it, it was still unclear to me whether ETL tools do support Data Vault out-of-the-box like slowly changing dimensions or not. So, who knows the answer(s)? The vendors?

3. Jos van Dongen – Confused by your question: imho ETL tools are for ETL, data modeling tools are for data modeling. You ask for ETL tools that support DV modeling. So strictly speaking: none . But out of the box support for DV (to some extent, depending on whether you actually want to generate a model from the source system) can be found in BI-Ready, Quipu and Wherescape 3D (still in beta); at Antonius Hospital we developed a Pentaho Kettle framework for loading a DV.

4. Juan José van der Linden - Hi, I don’t think there are ETL tools that support loading a Data Vault out of the box with special steps/transformations. Pentaho Kettle supports adding new steps as plugins. So if someone develops steps for loading hub/link/sats it can be made Data Vault aware. Quipu and BI-Ready support generating a Data Vault model and loading code from a datamodel. This model can be a business model or a source model. BI-Ready generates SQL as loading code. Quipu currently generates by default SQL code. The code generation of Quipu is template based. There are already users of Quipu building templates that generate Pentaho Kettle ETL transformations for loading the Data Vault. Regards, JJ.

5. Marco Schreuder – Compared to Dimensional modeling Data Vault loading is very simple: 95% inserts. You can use a combination of a lookup transformation to determine new rows and a data transformation task to inserts them. For satellites you have to determine the old rows and enddate them. You can use a simple SQL execute task for this. So… you really don’t need a dedicated component.

6. Ronald Damhof - Agree with marco. My simple answer: all do Nuff said

7. Juan José van der Linden – Loading hubs and links is always easy in a Data Vault. Also loading a satellite when there is only 1 change per business key per load cycle is also easy. It becomes tricky when you’re loading from a CDC system and have more changes per business key per load cycle or when you have a bi-temporal satellite. But even then a template system helps you to speed up developing.

8. Erik Fransen – Only those ETL-tools that are used by qualified DWH-designers are able to deal with Data Vault and data marts schemes and related data logistics correctly.

9. Daan van Beek – @jos you’re right, but why do almost all ETL tools available in the market support star schemas with slowly changing dimensions? That’s because it should make the life of a DWH developer much easier.

@marco @ronald the same holds true for star schema’s, 95% inserts. In my view ETL tools are created and used to get rid of hand coding SQL? Are you suggesting, just load a data vault with SQL and forget about ETL tools?

10. Erik Fransen – @Daan that’s what we are doing at our customers, and it works great. No need for expensive ETL anymore, but the right skills and holding on to your architecture-rules is crucial.

11. Martijn Evers – @Daan, You have to separate 2 different types of ETL steps:

1. Formal (semantical equivalent) model (to model) transformations (e.g. 3NF to Data Vault)
2. Pure data transformations (cleaning & matching, conforming etc).

Model transformation steps change the model but leave the data as is, while data transformations steps change the data (sometimes significantly) but do not change the model/shape of the data. ETL tools can do both, and usually in one BIG push. Most ETL tools however are not very smart on model to transformations (exceptions: Kalido, BIReady, Wherescape Red) and hence deliver poor ROI there. Most Data Vault (Oriented DWH) Architectures by default separate these 2 types of ETL actions (They should IMO) to accomplish an effective ETL divide & conquer strategy. Hence they can use different and far simpler tooling like (somewhat complex) metadata driven SQL or even Business Rule Engines to accomplish their task With Data Vault oriented DWH Architectures you can still use (powerful) ETL software but their weakness in model 2 model transformations and their focus on handling big and complex ETL steps (of any kind) is seen more as a liability then as an asset. Esp. in combining model transformations and data transformations, which can compromise things like reusability, maintainability, auditability & performance.

12. Daan van Beek – @Erik in general ETL tools would letting us lower the costs of data warehouse projects and maintenance by a factor of 3 to 5, are you so to speak sending all the ETL developers back to the stone age @Martijn, a distinction that is helpful, but a transformation for slowly changing dimensions (formal) is almost available in all ETL tools, why do the vendors not embrace Data Vault? Are users not asking for it? Or is it to complex?

13. Ronald Damhof - @daan; pls send met the research that states that ETL tools lower the costs of DWH projects & maintenance by a factor 3 to 5. Never seen that kind of research, software is not responsible for lower project or maintenance costs – it never is – so pls link.

@daan; “why do vendors not embrace Data Vault”, since when is normal-sense modeling something that needs to be supported by ETL vendors? Do not get it. I was not kidding when I said all ETL tools support Data Vault. I suppose you have researched the typical ‘complexity’ of DV mappings in ETL – can you show us what you find complex?

@daan; “users not asking for it” – that made me laugh.

14. Martijn Evers - @Daan, Except for slowly changing dimensions, of which DV uses a specific and simple case for loading satellites, there is not a lot of DV specific work a classic ETL tool can do, unless it can also do model transformations, which is usually not the case. Current ETL tools just can’t provide a very good ROI on Data Vault, so you can skip them in favor of simpler approaches like generated SQL.

15. Daan van Beek – @Ronald, “software is not responsible for lower project or maintenance costs – it never is” – really confused, ‘all’ increase in productivity of the last twenty years or so is mostly due to automation (for the last centuries mainly due to technology), and you are saying software (in combination with hardware) never can’t held be responsible for that?? Why do you think that?

@Martijn, thanks for giving a quick reply to @Ronald “current ETL tools just can’t provide a very good ROI on Data Vault”

16. Ronald Damhof – @daan; lol – not gonna get that research, am I?? You mentioned about research regarding ETL tools (not automation in general, you can automate ETL without ETL tools)….show me – you got numbers – factor 3 to 5 according to you in DWH projects and maintenance. I am curious what research you are referring 2. If you provide me the research I will answer your question – ok? @daan; all ETL tools support DV – the statement of Martijn does not change a thing to that statement

17. Daan van Beek – @ronald, Where is this going! You see things that aren’t there. I didn’t mention the word research at all, it’s common sense like we know that bikes (SQL) can’t ever beat cars (ETL tools) or a Ferrari (PowerCenter for example) in terms of speed, functionality and usability. And by the way, you said earlier in this discussion, ‘all do’. It would be fair if you try at least to prove your claim.

Please feel free to comment.

After users become familiar with your BI project’s benefits, they’ll likely want more. Be prepared to provide analysis of unstructured data. We’ll show you how to begin.

5/28/2008 by Kirby Lunger

You’ve spent the last five years defining, establishing, and building an analytical environment for your organization. You received accolades for finally providing access to structured information from your company’s transactional systems through a business intelligence (BI) tool with underlying data marts, a data warehouse, and a data integration tool. Now — all of a sudden, it seems — your colleagues are asking for access to other kinds of content such as e-mail, documents, and audio-visual media through your analytical architecture so they can use this content for predictive analytics in the BI application. Where should you start?

Content Definitions
To help your coworkers access this “unstructured” content, you first need to understand the types of data they want to retrieve. You probably have a good handle on the traditional transactional data that is housed in your analytical environment, especially the information in your databases (stored using multidimensional, relational, and other legacy formats). What your colleagues are asking for is pretty much all of the other content in the organization, which could make up as much as 80 percent of all corporate information assets. Users also want the ability to analyze content about the organization which is traditionally only available “outside the firewall!” This unstructured content is less ordered in terms of its information hierarchy, but the information is just as valuable for your performance management application. TDWI has described two types of data sources: semi-structured and unstructured. Semi-structured data includes spreadsheets, flat files, XML documents and RSS feeds. Unstructured data inside your organization is everything else you can imagine, such as e-mail, word processing files, audio-visual content, Web pages, and text fields in all of your organization’s applications. You may also be asked to provide access to, and analysis of, information outside your organization. This content comes in similar formats but is used for different business purposes.

Analysis Types
Your colleagues need access to this semi-structured and unstructured content to answer questions such as What are our company’s contractual obligations across the enterprise? Is our organization meeting its compliance reporting requirements? Your coworkers may also want to gain access to content created outside of your company to address questions such as What do our customers think of our products and services? What are our competitors doing? What trends and buzz in the marketplace could influence our organization?

Architectural Approaches
The most commonly accepted approach today is to use textual analytics and/or extract, transform, and load (ETL) software to impose order on a data set that may be comprised of many different types of data. These tools deconstruct textual content (often using natural language processing) into data about specific, defined items such as customers or products. These items then are translated into a traditional data structure, such as records in a database row, or entities in a hierarchy. This approach provides some clear advantages. The most obvious is that this content can be integrated into your current BI environment for presentation and analysis. This is a good incremental step if you are trying to get a sense of what value you can derive from this kind of analysis. However, this is only an incremental step to a more fully featured solution. This approach only addresses a subset of the semi-structured and unstructured data, and does not provide new analytical tools for exploring combinations of structured, semi-structured, and unstructured content. A more radical and interesting approach to this problem is to apply techniques that are not commonly used in a more structured environment, primarily from the search software field. Today, we generally assume that when a user accesses the BI platform, he has a precise query in mind such as when your sales manager asks, “What is the most recent Region One sales forecast and how does it compare to actual sales?” In contrast, most search platform users are not totally sure what they are looking for. They may have a few parameters in mind (e.g., my new car should be red and have a high-safety rating), and they need the search platform to help them find information that is as relevant as possible to their query. The sales manager using a search paradigm might ask, “I see that performance was off in Region One last quarter. What were the causes of performance decline in Region One last quarter?” Such “fuzzy” search logic implies a different approach to integrating your semi-structured and unstructured content into your analytic platform. Rather than looking at text as your only data source, you need to provide access to all types of content. This means that rather than folding this content into your data warehouse through an ETL process, you may need to consider some of the newer content ETL products just introduced for this type of initiative. Data visualization and presentation also evolve when you take a more “search”-focused approach to such content. In its simplest application, this means searching in your BI application, optimized for the content your users will be trying to search. In a more complex scenario, you could offer analysis techniques such as a content terrain (or “heat” mapping) which is similar to a regular topographical map: this visual technique demonstrates content clusters based on particular areas of concentration within your enterprise.

Getting Started
To start down this path, you will obviously need to take a more holistic view of your organization’s information and technology architecture to learn what data is available to your end users. You also need to spend time learning what is missing today from the BI environment. Don’t be surprised if people at first cannot articulate their needs in this arena — most people do not believe current tools can support this analysis!

Source and more information: Enterprise Systems Journal

By Joy Mundy

The Extract, Transformation, and Load (ETL) system is the most time-consuming and expensive part of building a data warehouse and delivering business intelligence to your user community. A decade ago the majority of ETL systems were hand crafted, but the market for ETL software has steadily grown and the majority of practitioners now use ETL tools in place of hand-coded systems.

Does it make sense to hand-code an ETL system in 2008, or is an ETL tool a better choice? Kimball Group now generally recommends using an ETL tool, but a custom-built approach can still makes sense. This article summarizes the advantages and disadvantages of ETL tools and offers advice on making the choice that’s right for you.

ADVANTAGES OF ETL TOOLS

Visual flow and self-documentation.The single greatest advantage of an ETL tool is that it provides a visual flow of the system’s logic. Each ETL tool presents these flows differently, but even the least-appealing of these user interfaces compare favorably to custom systems consisting of stored procedures, SQL and operating system scripts, and a handful of other technologies. Ironically, some ETL tools have no practical way to print the otherwise-attractive self documentation.

Structured system design. ETL tools are designed for the specific problem of populating a data warehouse. Although they are only tools, they do provide a metadata-driven structure to the development team. This is particularly valuable for teams building their first ETL system.

Operational resilience. Many of the home-grown ETL systems I’ve evaluated are fragile: they have too many operational problems. ETL tools provide functionality and practices for operating and monitoring the ETL system in production. You can certainly design and build a well instrumented hand-coded ETL application, and ETL tool operational features have yet to mature. Nonetheless, it’s easier for a data warehouse / business intelligence team to build on the management features of an ETL tool to build a resilient ETL system.

Data-lineage and data-dependency functionality. We would like to be able to right-click on a number in a report and see exactly how it was calculated, where the data was stored in the data warehouse, how it was transformed, when the data was most recently refreshed, and what source system or systems underlay the numbers. Dependency is the flip side of lineage: we’d like to look at a table or column in the source system and know which ETL modules, data warehouse tables, OLAP cubes, and user reports might be affected by a structural change. In the absence of ETL standards that hand-coded systems could conform to, we must rely on ETL vendors to supply this functionality — though, unfortunately, few have done so to date.

Advanced data cleansing functionality. Most ETL systems are structurally complex, with many sources and targets. At the same time, requirements for transformation are often fairly simple, consisting primarily of lookups and substitutions. If you have a complex transformation requirement, for example if you need to de-duplicate your customer list, you should use a specialized tool. Most ETL tools either offer advanced cleansing and de-duplication modules (usually for a substantial additional price) or they integrate smoothly with other specialized tools. At the very least, ETL tools provide a richer set of cleansing functions than are available in SQL. Consider buying the ETL & Data Integration Tool Survey to see how the ETL tools compare on this aspects.

Performance. You might be surprised that performance is listed last under the advantages of the ETL tools. It’s possible to build a high-performance ETL system whether you use a tool or not. It’s also possible to build an absolute dog of an ETL system whether you use a tool or not. I’ve never been able to test whether an excellent hand-coded ETL system outperforms an excellent tool-based ETL system; I believe the answer is that it’s situational. But the structure imposed by an ETL tool makes it easier for an inexperienced ETL developer to build a quality system.

Source and more information: Intelligent Enterprise

Most of the organisations polled (68%) report that data volumes have increased substantially over the past three years, with a further 25% indicating more modest rises. Only 2% reckon that data volumes have stayed constant over that time period.

Half (54%) of the sample have already initiated data warehousing projects, with 10% currently evaluating their options and 22% considering such a development. One in five has started a programme within the past year, while a quarter (24%) have had a data warehousing programme for the past two years and the same proportion (24%) for between three and five years.

The main reason for implementing a data warehouse emerges as the desire to improve the flow of management information, cited by 61%. However, the accuracy and reliability of corporate information remains a crucial issue for many, with 28% admitting that data quality was ‘bad’ prior to implementing a data warehouse.

Just 10% describe their data quality as ‘good’ before implementation, and none believe it was ‘excellent’. In contrast, after implementing a data warehouse, the majority (45%) now report that data quality is ‘good’ and 6% say it is ‘excellent’.

Making improvements on this scale takes time, with 16% of companies spending up to 12 months cleaning up their data and 13% more than a year. There are clear benefits from this exercise, though, as 48% now report no difficulties in analysing the data obtained from their data warehouse.

This metaphor – the information supply chain – is in some ways a simple extension upon thoughts already well developed by others, most notably Bill Inmon’s Corporate Information Factory (CIF), and yet the supply chain metaphor doesn’t seem to enjoy much use in the marketplace, at least not as a framework for holistically describing both the DW and Business Intelligence, together.

Inmon describes the enterprise data warehouse and the various satellite systems that surround it – operational data stores (ODSs); extract, transform and load (ETL); data marts, etc. – as similar to a brick-and-mortar factory. The factory inputs many disparate raw ingredients (data from all the company’s operational systems), subjects those raw data inputs to a series of processes wherein the data is refined and integrated (ETL) and finally spits out a finished good on the other end (corporate information). Whether a practitioner leans more toward Kimball’s dimension bus architecture or Inmon’s CIF as the guiding vision for how to build a data warehouse (or like this author you feel the two approaches are more complementary than contradictory), Inmon’s metaphor is simple to understand and helpful for explaining what data warehousing is all about.1

Extending the metaphor to encompass the field of business intelligence is beneficial, since definitions of Business Intelligence seem pretty slippery, and it can be unclear how Business Intelligence overlaps or is different from the enterprise data warehouse. I believe the reason for this confusion, at least in part, comes down to how the term “BI” is often used – that is to say – it is used to convey purpose, rather than function. Business Intelligence is typically defined as: the collection of Business Intelligence tools and techniques used to help companies make better decisions. This definition is very broad. It expresses a worthy purpose, but it casts a huge net within which almost anything can fall and thereby blurs important distinctions.

Read the full article: DMReview.com

bMighty: What is it about the current business climate that makes mining data for BI about a company’s customers so vital?

Linoff: Once upon a time–and this is a rosy view of the ancient world–the hardware store owner knew you. If you went to the grocery store, someone there knew what items you needed. If a woman walked into a bank to make a deposit and she was pregnant, [the bank manager] knew and offered her a car loan. We traded all that for lower interest rates and lower prices, we traded all that for more efficiency, but we lost the intimacy with our customers. How can we get to know our customers again? That’s one component of business intelligence.

Where do small and midsize companies fit into this BI picture?

Well, it depends upon their relationship with their customers. If they only have a handful of larger clients, they know them. But one of our clients, The Vermont Country Store, is a cataloger that sends to several million people. The company has about 400 employees. How are 400 people going to know about several million people? They need to use the information they collect in order to better serve their customers and increase revenu.

But there’s not an overall approach, because businesses are different, and there are several different levels that businesses operate on. Some are campaign-driven, to encourage customers to buy, and [business intelligence helps] them optimize each campaign and [some use] traditional predictive models associated with data mining, tailoring aspects of their Web sites to their customers or sending e-mails to their customers to return to their site.

For example, on Amazon.com, which is not a small company, if I bought Harry Potter IV, [they know] I will buy Harry Potter V, VI, and VII, [because they're asking] what’s the next type of thing that I’ll buy? You can get associations or market basket analysis. For companies in retail, association rules are something they’re interested in. What things sell together?

How can a company’s IT managers maximize the benefits of business intelligence?

The IT manager needs to enable communication–it’s the most important thing. In most companies, when [management] asks questions, [the IT manager] says it can’t be done, because people get trained not to make decisions off of data. For instance, if the marketing department of a company wants to plan a promotion, and needs to know what other items sold last year when soup was on sale, the IT manager might say, “we’ll know in three months.” So the question stops being asked and opportunities are lost. Then people start making decisions based on intuition and they [lose] the ability to make more informed decisions.

How can they get that ability to make more informed decisions?

It’s the people on the front line–the worst-paid people–who know the customers. You need a mechanism to get that information out. I have another great example. There was a guy who owned a chain of drugstores that sold chocolates on Valentine’s Day. It was big business for them. Using their BI system, the SAS OLAP server, they sliced and diced their data to discover that two of the stores had higher sales in chocolate. Both were managed by the same person.

They discovered that instead of placing the lower-priced chocolates on a lower shelf and the more expensive chocolates higher up he was mixing them together. It turns out, guys were coming in and reaching for the cheaper stuff and then saying, “she’s worth it.” This demonstrates that people on the front line know what’s going on, so you need to have systems to detect what they know. You need systems in place to communicate the best practices.

Source: Forbes ]]> http://www.etltool.com/how-to-mine-all-that-customer-data/feed/ 0