Automotive Digital 3 : Big Data Analytics and Advanced Computing

This entry is part 3 of 4 in the series Automotive Digital

Click on images to enlarge.


In 2017, Ford Motor Company hired its first chief data and analytics officer. So too did General Motors. Why wouldn’t they? After all, both have years of experience with connected vehicles. In 1996 GM factory-installed On-Star  and Ford installed Sync so, arguably, they led the way towards the connected car. But, despite their head start neither has a distinct competitive advantage in the connected car space.

Why was that? Possibly they viewed production and engineering capacity and global distribution expertise as their main strengths. But, if that was true then, it’s no longer true now. That view changed after 2015.  All global  Auto OEM’s recognise that ‘CASE’ describes their emerging, linked, competitive arenas: Connected, Autonomous, Shared and Electric.

VW Data Centre operated by Verne Global in Iceland

Pivoting towards CASE has brought rapid change. In 2016, after 64 years, Ford began to restructure its Research and Engineering campus outside Detroit. By 2017 it had announced a $200MN investment in its own data centre, anticipating a 1,000% increase in data volume due to connected cars. BMW announced its own data centre in 2017. Volkswagen announced it’s move to a data centre in Iceland in 2016 and, in 2020, VW opened a further two data centres in Norway. Daimler has been creating and managing its own data centres since 2008 with assistance from Fujitsu and others.

By 2020 it was clear from their press releases that most Auto OEM’s recognise that, similar to laptop computer and mobile phone makers, building great hardware will only allow them to survive. Building and deploying great software will allow them to thrive. It’s this realization that underpins their race to transform themselves from physical into digital firms. They need to create the software to handle ‘big data’ not just bolt on the parts that handle it

If managing large quantities of data is on their agenda, so too are the computing tools needed to analyse it. Two in particular are attracting significant investment: quantum and edge computing. VW, Bosch, Ford, BMW and Daimler are all investors in this space because they see the end-use cases in applications such as city-wide traffic management and production logistics. But it was only in 2019 that the EU earmarked €1BN in a ten-year investment plan for quantum technologies. US venture capital firms had already invested $1BN plus between 2016 and 2019, so Europe will be playing catch-up for some time. And advanced computing is already moving beyond quantum. Have you heard of DNA computers? If not, watch this video and see how Boston-based startup Catalog Technologies Inc. recently showed it could store 14 gigabytes of data from Wikipedia in DNA molecules, which look like coloured water, in a bottle.

What is Big Data?

Big Data usually refers to data sets that are too large and/or too complex to be handled by traditional relational database processing software on a single computer, even a single super-computer. Taking vehicles as an example, increasing numbers of installed micro-processors, sensors and internet connections mean that connected cars and trucks have the potential to generate, store and transmit copious amounts of real-time data. Some estimate 25Gb per day of data on the vehicle’s speed, location and technical condition could be available. Added to that will be data sent to the vehicle about traffic conditions, optimal routes and services. Even a super-computer would find this volume of data arriving daily from a large fleet of vehicles overwhelming. This is what is meant by ‘Big Data’.

What sets ‘Big Data’ apart are five characteristics: volume, velocity, variety, veracity and value. Big data is large volume; terabytes or greater of data per second or per hour. It is generated at high velocity – continuously in real-time. It involves a variety of data sources – audio, video and text and may be structured or unstructured. Its veracity can be relied on. It is complete, accurate, current and precise so it can be used with confidence. Big data has practical value. It can give insights that reduce cost, speed processes and get to the right answer faster

What has excited…and worried… business about Big Data is its potential to create new services, markets and financial opportunities. If data can be collected, stored and analysed in a timely manner and at low cost, new businesses can be created. For example, how many vehicle or fleet owners would be prepared to pay more to shift from scheduled to predictive maintenance? It would benefit manufacturers, dealers and users alike. How many insurance companies would be willing to pay for real-time data on driver behaviour? There are many more possibilities: Improved fuel efficiency and driver assistance alerts could reduce crashes and inconvenience; The vehicle could become a transaction point for ordering services, from entertainment to hospitality; It could enable tailored marketing messages based on driver characteristics and behaviour; Fleet operators could find ways of reducing their running costs and damage; ‘Smart’ cities could reduce traffic congestion, accidents and pollution levels with targeted messages and vehicle tracking.

But, opportunities for one firm may be threats for others. In 2020 Auto OEM’s employees and distributors are still coming to terms with the implications of switching from an ownership-based to a subscription service based business model. But competition is fierce. Tesla launched its autonomous driving software as a subscription service in June 2020, threatening to up-end traditional OEM’s ambitions to restore profitability through premium pricing for this capability.

What is Big Data Analytics?

Many of us are familiar with the basics of data analytics. We use spreadsheet software, such as Open Office or MS Excel, to enter data and perform queries on that data, often in the form of calculations. Some users progress to data bases or even further, to data visualization, using interactive graphs or charts. Even they use only a fraction of the capabilities of the software packages. And, why shouldn’t they? Most are practical people whose time is valuable, so they develop just enough software skills to finish the tasks that they regularly complete.

In terms of computing, self-managed spreadsheets, data bases and visualization are relatively simple ‘front office’ tools. Big data analytics are ‘back office’ processes and techniques that are steps beyond in complexity. To begin with, they use specific programming languages, such as Python, SPSS and R, to write the software. Second, they use specialized infrastructure for distributed programming, such as Hadoop and Spark. Third, they’re integrated with cloud providers as a platform for development and distribution. So, large data sets use special programming languages and  structuring tools used to organize and process the data. They take huge amounts of data, organize and analyse it.

Types of Big Data Problem Solving

That’s all? No. They don’t only analyse different types of data – text, numbers, video, audio and images – at almost real-time speed. They also use advanced analytics techniques such as text analytics, artificial intelligence, machine learning, predictive analytics, data mining, statistics and natural language processing to gain new insights from previously untapped data sources. They create new knowledge.

What is Advanced Computing?

In the realm of ‘Big Data’, advanced computing is looking to innovations that take the shortest time taken to process data. Two are already promising for auto OEM’s: one is Quantum Computers and another is Edge Computing. Both are integral to the ‘CASE’ vision of the future path of automotive development.

What is Quantum Computing?

I’ll keep this brief, for a simple reason. I’ve never seen a quantum computer for real. But I do know the basic theory and I’m amazed at their potential. The digital computers that you and I use encode bits as ‘1’ or ‘0’. On or Off. There is no in-between. The choice is binary. Your favourite video or song track is simply a long string of zeros or ones in a specific sequence. Physically computer ‘bits’ are electrical signals passing along a computer’s circuits.

Quantum computers use ‘qubits’. Qubits are a different physical form from conventional ‘bits’. They are made of sub-atomic particles – photons and electrons. This gives them two important characteristics  that conventional ‘bits’ do not possess. First, they do not have to be just ‘1’ or ‘0’. They can be both at the same time. At least, they can be until someone measures them when they collapse into zero or one. This ability to be in two states at one time is called ‘superposition’. Second, when two qubits are linked – called entangled – a change in one qubit will trigger a change in the other one in a predictable way. This will happen even if the entangled qubits are distanced apart. Einstein famously described it as “spooky action at a distance”. Researchers still do not fully understand why entanglement occurs but the implications are clear. In a conventional computer, doubling the number of bits doubles its processing power. But thanks to entanglement, adding extra qubits to a quantum machine produces an exponential increase in its number-crunching ability. However, all is not simple in the world of quantum computers.

What’s the catch? Well, at the moment, they’re still very fragile and error-prone. Qubits can tumble out of superposition due to environmental effects. As a result, they’re maintained in supercooled fridges inside a vacuum chamber.

Who’s interested in quantum computers? IBM, Microsoft and Google. Banks like J P Morgan-Chase. And of course, auto OEM’s, such as Volkswagen and Daimler. What’s their interest? Daimler is looking into how QC could potentially supercharge AI, plus manage an autonomous-vehicle-choked traffic future and accelerate its own logistics. Volkswagen too, is working on streamlining traffic flows in Beijing, Barcelona and Lisbon.

What is Edge Computing?

Centralised V2C Infrastructure – overwhelmed with data AECC 2019

According to the Automotive Edge Computing Consortium (AECC) Edge computing is the science of having the edge devices in the vehicle, or local to the vehicle, handling computations themselves. This reduces the need for data to be transported to another server environment… and that speeds up the processing time.

It means that the sensors inside and outside the vehicle process the data they collect, share timely insights within and outside the vehicle and if applicable, take appropriate action. Limitations to Cloud capacity are driving this approach to infrastructure architecture.

When connected cars were envisaged, Auto OEM’s forecast 1GB  of data per vehicle per month by 2025. The current estimate is 1,000 times higher per vehicle by 2025. With that data load, when new services are developed and deployed, the cloud infrastructure would be unable to upload it or send back answers fast enough.

Edge Computing – Distributed Processing Infrastructure AECC 2019

The alternative is to process data within the vehicle and have a hierarchical data processing infrastructure. Distributing navigation software updates to vehicles overnight is a perfect task for cloud computing. But, the decision of whether to veer left or right to avoid a truck is faster made independently by an onboard computer – it’s certainly not the time to wait for a server in a remote data centre to respond. Between these two extremes, infrastructure is in place, such as networked traffic-light systems and mobile phone masts, that could be used as computer processors at the edge of the network. For example, if a driver is speeding the wrong way down the road toward you, telling your car which way to veer off the road – without colliding with the pedestrian on the hard shoulder – is something you would like to happen in  milliseconds. In a life-or-death situation, you want the data processed right now in real-time so that you can avoid the collision while there is still a possibility of saving your life. An OEM’s data centre 2,000 miles away is just too slow.

How this affects automotive?

Imagine you’re an auto OEM and you want to analyse a new material. It’s molecule has 41 atoms. To do it you need a supercomputer with 1086 bits. That computer would require more transistors than there are atoms in the observable universe. But for quantum computers, this type of simulation is well within the realm of possibility, requiring a processor with 286 quantum bits, or qubits. The same is true for optimising global logistics or transportation systems. VW are using Quantum Computers to optimise traffic flow in Lisbon.

What these uses illustrate are the possibilities for integrating these digital technologies: Big Data and Analytics + Advanced Computing + Machine Learning + Artificial Intelligence. Integrating  offers a bright future for Auto OEM’s and Distributors but the outcome is far from certain. No one in automotive is a ‘digital native’. There are extremely large, well-funded competitive businesses outside automotive that are. The prize will go to the swiftest.

So, wherever you are in automotive take a look at your business processes. Is there data that you think is not worth collecting because it seems unmanageable? Can you see how you could use it in the future, if you could solve the technical problems? If you did, would it ensure an ultimate competitive advantage for your business? If the answer to some of these questions is yes, it’s time to take a deeper look at the new digital tools.



Series Navigation<< Automotive Digital 2 : Blockchain and Smart ContractsAutomotive Digital 4 : Smart Cities, AR, VR, Robotics and Additive Manufacturing >>