AIAI Data Mining

Introduction | Technical Background | Applications | Tools | Websites | Mailing lists


Databases can contain vast quantities of data describing decisions, performance and operations. In many cases the database contains critical information concerning past business performance which could be used to predict the future.

Often the sheer volume of the data can make the extraction of this business information impossible by manual methods. Data mining is a set of techniques which allows you to do this.

Data Mining (also known as Knowledge Discovery) technology helps businesses discover hidden data patterns and provides predictive information which can be applied to benefit the business.

The basic approach is to access a database of historical data and to identify relationships which have a bearing on a specific issue, and then extrapolate from these relationships to predict future performance or behaviour. The human analyst plays an important role in that only they can decide whether a pattern, rule or function is interesting, relevant and useful to an enterprise.

Technical Background

Data Mining and Knowledge Discovery in Databases are terms used interchangeably. Other terms often used are data or information harvesting, data archeology, functional dependency analysis, knowledge extraction and data pattern analysis. A high level definition of Data Mining is: the non-trivial process of identifying valid, novel, potentially useful and ultimately understandable patterns in data. Data mining is not a simple process and there is no tool that can do the job automatically. Data mining can be aided by tools, but it requires both human data mining expertise and human domain expertise. Data mining consists of a number of operations, each of which are supported by a variety of technologies, such as rule induction, neural networks, conceptual clustering. In real world applications information extraction requires the cooperative use of several data mining operations and techniques.The basic data mining process is as follows:

  1. Define the business objective and expected operational environment of any expected resulting system.
  2. Select data. Substantial databases with a meaningful sample of data is required. Selecting data often consists of selecting a time span, geography or product set that you want addressed, and the variables that you want to consider.
  3. Transform data. This involves determining how to represent the data for the data mining algorithm, e.g. age, should it be represented as a member of a set (25-35 year olds) or a straight number.
  4. Run the data mining algorithm or combination of algorithms. Iteration to step 3 or 2 is often needed here. The AI techniques of rule induction and neural networks are used for this machine learning stage.
  5. Analyst examines the output data. Often visualisation plays an important part in helping the analyst, especially if the analyst needs to present their analysis to others.
  6. Present results to the business, in order that the insights can be incorporated into business processes (e.g. through producing output data files or installing data mining software).

Data mining is typically not used as a business system delivery technology. Rather it is an extremely powerful and effective set of technologies for analysing and clustering data which can be used to form the basis of a system.


The key reason why Data Mining is such a buzzword at the moment is that because many organisations recognised the need to better understand their customers. Data mining can deliver real world results. Data mining has been used for the following types of applications:


The July 1996 Edition of "Intelligent Software Strategies" was devoted to Intelligent Data Mining Tools . They classified the tools by their knowledge discovery techniques. The 7 classifications were: Rule and Decision Tree Discovery; Neural Networks; Conventional Statistical; Advanced Visualisation; Fuzzy Techniques; Knowledge Based; and Multiple Techniques. Over 40 tools are reviewed and addresses of the vendors (some with website addresses) are given.


Mailing Lists

Index | About AIAI | People | Projects | Technologies | Business Services | Publications | Info. Services

Last updated 2nd June 1997
by Ian Harrison