Data Warehousing and Data Mining B.Sc. CSIT 7th Semester

Challenges of Data Mining

In several sectors, data mining and knowledge discovery is becoming a critical technology for businesses and researchers. While data mining is becoming a well-established and reputable subject, there are still numerous difficulties to overcome.

Some of the challenges are:

Mining methodology and user interaction issues

 It refers to the following kinds of issues 

Mining different kinds of knowledge in databases 

Different users may be interested in different kinds of knowledge. Therefore it is necessary for data mining to cover a broad range of knowledge discovery task.

 Interactive mining of knowledge at multiple levels of abstraction 

 The data mining process needs to be interactive because it allows users to focus the search for patterns, providing and refining data mining requests based on the returned results.

Incorporation of background knowledge 

To guide discovery process and to express the discovered patterns, the background knowledge can be used. Background knowledge may be used to express the discovered patterns not only in concise terms but at multiple levels of abstraction. 

Data mining query languages and ad hoc data mining 

 Data Mining Query language that allows the user to describe ad hoc mining tasks, should be integrated with a data warehouse query language and optimised for efficient and flexible data mining. 

Presentation and visualisation of data mining results 

 Once the patterns are discovered it needs to be expressed in high level languages, and visual representations. These representations should be easily understandable. 

Handling noisy or incomplete data 

 The data cleaning methods are required to handle the noise and incomplete objects while mining the data regularities. If the data cleaning methods are not there then the accuracy of the discovered patterns will be poor.

 Pattern evaluation 

The patterns discovered should be interesting.

Performance issues

 Efficiency and scalability of data mining algorithms 

In order to effectively extract the information from huge amount of data in databases, data mining algorithm must be efficient and scalable. 

 Parallel, distributed, and incremental mining algorithms 

Parallel and distributed data mining algorithms divide the data into partitions which is further processed in a parallel fashion. Then the results from the partitions is merged. The incremental algorithms, update databases without mining the data again from scratch.

Diverse Data Types Issues

 Handling of relational and complex types of data 

The database may contain complex data objects such as multimedia data objects, spatial data, temporal data etc. It is not possible for one system to mine all these kind of data. 

Mining information from heterogeneous databases and global information systems 

The data is available at different data sources on LAN or WAN. These data source may be structured, semi structured or unstructured. Therefore mining the knowledge from them adds challenges to data mining. 

Also Read: Knowledge Discovery in Database

About Author

Sarina Sindurakar