Breaking the Enigma

Originally deciphered by the team at Bletchley Park in the 1940s, Enigma Pattern instead used a range of modern machine learning and artificial intelligence techniques and methodologies to break the code in record time – and at a cost of only £10 – by using cloud services platform DigitalOcean’s powerful cloud servers, known as Droplets.

The original Enigma machines were electro-mechanical rotor cipher machines used by Germany and its allies during World War II. They were portable, resembled typewriters, and since they encoded messages' letters with an ever-changing polyalphabetic cipher, were considered hack-proof and used as the primary communication system, transmitting orders and reports.

For a long time it was considered impossible to break the code. The user needed the original message and a password which were then used in combination to perform some very complex letter substitutions. The algorithm organising the substitutions was sophisticated so that even if the enemy had captured both the machine and the encrypted message, it could not be decoded without the password. The Polish came up with the idea of building automatons that mimicked the letter substitutions and tried all the possible variants of the password. Firstly they and then the British created bombes –the size of trucks metal racks - filled with buzzing rotors, where the passwords were tested. Since the number of possible combinations was extensive, and certainly too large for the bombes to test them in any reasonable time, the mathematicians worked hard on limiting the password variants to a more manageable number. The British were more successful. Using machine speed and human intuition finally led to cracking the code, even for the most sophisticated U-boat (German submarine) type of Enigma. It is estimated that cracking the code saved the lives of 14 to 21 million people and shortened the war by two years.

Enigma Pattern’s team, led by Lukasz Kuncewicz, decided to break the war relic using modern techniques, as a demonstration of how far artificial intelligence has come. First, the team taught the artificial intelligence system to recognise the German language by feeding it Grimm’s fairy tales. After long hours contemplating stories and the words, the system started to become more accurate in its classification of language. They then recreated the most sophisticated version of Enigma (four rotors navy type, one pair of plugs), which has 15,354,393,600 password variants, in the programming language Python. Just like the bombes sed by the Polish and British, they set it up to test all possible combinations of the password – the only difference being they did not limit the number of passwords.

The output of this bombe was connected to the artificial intelligence system, so if the simulator, testing all the possible password combinations, was sending something resembling the German language, the artificial intelligence signalled it as a properly decoded message.

Lukasz said: “Enigma code is a complex system and decoding the message takes time as does checking to tell if the decrypted message is German. When you have billions of possible passwords to try, it all adds up to days. If we had used the power of only one machine, it would have taken 2 weeks to go through all of the possible combinations.” The team at Enigma Pattern realised that it needed to shrink the time taken to decode the message and turned to DigitalOcean, a cloud services platform designed for developers and businesses to build, scale and automate their infrastructure. “There are a few cloud services providers that let you hire machines immediately and for the time you need, be that one hour, several days or longer-term,” said Lukasz. “One of these is DigitalOcean, who we contacted and asked if we could spin our project on 2000 of their virtual servers. They gave us the green light, we prepared a parallel version of our bombe, and it worked!”

At the event in central London, by using DigitalOcean’s virtual servers, the entire Enigma machine decryption was finished in just 13 minutes, with 41 million combinations tested per second.

Alex Jaimes, Head of Research and Development at DigitalOcean said: “Things have moved on massively since the original Enigma machine was decrypted, but the fascinating element to Enigma Pattern’s work is how they’ve used the power of AI and cloud computing to give a glimpse into the future of everyday computing. The simplicity and computational power of our platform makes DO (DigitalOcean) the perfect partner for AI projects that require scale, speed, and execution at remarkably low costs.”

^{© Andy Hollingworth Archive and Simon Singh}

Tom Dent-Spargo speaks to the Engima Pattern team about the project and the future of AI

RLJ: What prompted the decision to crack the Enigma?

Mike Gibbons (Chairman of the Board and Co-Founder of Enigma Pattern): Our vision for Enigma Pattern is driven by the inefficiencies we see around us in the world today. There is considerable waste of people's time, food, energy and other resources. We want to use the techniques of Big Data, AI and Machine Learning to help improve the way we live and work. For many people Artificial Intelligence is a mysterious and sometimes frightening concept we were looking for a way to better explain it to people. We hit upon the idea of cracking the Enigma Code because firstly it is a well known challenge and secondly it is known as being a very tough problem to solve.

RLJ: What other applications are there for this AI?

Lukasz Kuncewicz (Head of Data Science at Enigma Pattern): The process of cracking the code used this setup: AI can answer a simple question – “Is this text German?” , “How German is it?” AI is rather patient, and you can continue to ask it billions of questions such as, “How about this text?”, “How about this?” it will not mind. This allowed us to crack the code. Now, the same setup can be used in many other applications.

For example, in Medicine, AI can predict if the condition of a patient will improve, it can also ask questions such as, “What would happen if we applied this treatment?”, “What if a different one?”, “If we use this dosage?”, “What if we use half of it?” etc.

In advertising, AI can predict if a user clicks on a banner. It can also consider questions such as, “What would happen if we used this banner?”, “What if the text was different?”, “What if the colour was red?”, “And if we put a button here?”, “Or to the left?” and so on.

RLJ: Does anyone need to worry about their banks being susceptible to an AI hack in the wake of this demonstration?

LK: Our version of Enigma had 32 billion of combinations, and in a binary system you would need only 36 digits to store all the possible passwords. The bank systems usually use at least 128 digits keys (passwords) – so even if you can test a quintillion (1,000,000,000,000,000,000) combinations per second, it still would take more than a trillion (1,000,000,000,000) years to crack the code. The worried wealthy can sleep peacefully.

RLJ: The dramatic lowering of the cost of digital power is staggering since the original Enigma. Are data centres such as DO the key to unlocking AI’s huge potential?

LK: In my opinion there wouldn't be AI revolution if there wasn't cheap computation power available to (even small) companies. It's not only about data centres like DigitalOcean (although they do make great work bringing cheap computation power to people), nowadays even your laptop has got sufficient power to be used as machine learning environment. Granted, the more data you have, the better (usually) are the results, so big companies like Facebook or Google are going to win when it comes to speech recognition or user behaviour modelling. But if you have some data in a small company and want to use machine learning to optimise ROI or cut on unnecessary expenses, your laptop probably will be sufficient. If not, you can always spend literally a few dollars on DO and have it cracked there.

RLJ: What's next for Engima Pattern?

MG: Enigma Pattern will continue to help organisations better understand their data and exploit the patterns hidden within. We approach this from a very practical point of view. We work first in an exploratory mode then once we have identified an area of potential interest we implement solutions to start bring results quickly. From then on it is an iterative process building on what we discover to delve further into the available information.

Background: Enigma Pattern and DigitalOcean

Enigma Pattern is based in the heart of Silicon Valley in San Jose and helps organisations maximise value from their data by supplementing the regular analytics processes with modern Machine Learning and Artificial Intelligence techniques and methodologies.

DigitalOcean is a cloud services platform providing computer, storage and networking solutions, Founded in 2012 with offices in New York and Cambridge, MA, DigitalOcean has one of the largest libraries of open source resources available.