Data mining is a tool that builds models to understand relationships and patterns from large databases. It is an umbrella term that brings together the fields of statistics, artificial intelligence and machine learning. Many people who work with data have used this technique for years, but the improvement of technology in the last decade has allowed for many more possibilities. More processing power has allowed for data mining to develop and provide more in-depth insights.
Data mining got its official name in the 1990s, but it is probably not until recently that the public has felt the impact. Most people will associate it with social media, especially after the publicised privacy scandals that have challenged social media platforms numerous times. This association could be considered unfair - data analysis is the method, not the result. To be able to use social media we agree to the terms and conditions, and in turn, provide them with data they can use. However, we do not grant permission for them to be careless with the information.
With this in mind, is it unethical? Unfortunately, it is hard to say, as it is dependent on both the data and whom it effects. For example, American supermarket Target used data mining to analyse women's shopping baskets and identified patterns of which items related to pregnancy. Target then sent vouchers for baby related items to other customers who had bought similar products. Many customers enjoyed this; some even found it comical that Target knew they were expecting a baby before they had formally announced this. The supermarket discovered the ethical problems that can occur from data analysis when they sent the vouchers to the home of a teenage girl. The girl was pregnant, so the algorithm was correct, but had not told her family, which caused them distress. Legally, the supermarket was not at fault; however, they may not have fully considered the ethical implications of the campaign.
Companies should not be afraid of using data mining, as long as it is used responsibly. It can be a great tool that can help to build marketing strategies, reduce costs and reach new customers. Every organisation will have pages upon pages of information that has no meaning, until someone analyses it. For example, a Travel Agent might know your relationship status, if you own your own home and your age - a pretty ‘useless’ list until it’s looked into further. A data scientist could look at this list and finds trends that show 34-year-old single females are more likely to travel to Italy than elsewhere in Europe. This information could be vital in ensuring the success of your next communication campaign.
If someone is looking to take data mining seriously, what attributes should a data analyst scientist have? A great start is a bachelor level qualification in maths, statics or computer science. Someone with strong communication skills is also invaluable, to be able to translate their findings. Finally, they should have a real passion for the industry, dealing with large quantities of data can be hard if they have no real interest in the work. Being enthusiastic about their role will make their findings better.
People are unlikely to be upset that a company is analysing their data, but are often upset when things go wrong. Businesses should be aware of their obligation and responsibilities of the data they hold. Security breaches should be reported to clients straight away and measures put in place to prevent this from happening again. Albeit more difficult, they should also consider the implications of their research, as some results can cause conflicts with their customers. Data mining has always been a fantastic tool to use, but in the modern world, it should be used with more awareness than some major companies care to show.