After two months of endless work, on a sunny spring morning PG achieved a big milestone. All the one thousand sensors where successfully connected to the IoT hub and were transmitting force vectors at a a rate of 1000 per second equating to 24MBS or 35GB per day. Wasn’t it anything other than a highly motivated cofounding team, that day would have been spent with celebration and opening champagne bottles but to John and Megan success was only defined as living in the skies. So right after a passionate hug, they began thinking about the next steps. Now they had to start analyzing the massive amount of data and build a model that could be used to specify the position of the floating car in the sky. They then could use this model to predict the object location based on the realtime data the received from sensors. Finally they needed to build simulations that went beyond just John’s Tesla and demonstrated how an entire city floating in the sky.
Megan started looking at options available on Azure for each of the problem at hand. First for building the model, Azure ML Studio seemed to be a simple tool but its limit of only allowing using 10GB to train data made it off the list. A solution that could work for them was to provision several data science virtual machine (DVSM) and use distributed computing to build and optimize the model on them.
The second piece required to predict the location of the floating objects using the model. For that they decided to go with PaaS premium scalable API service that performed 1000 computation a second using the 1000-dimensional force vectors they received from sensors and the model that was built is step 1.
The last piece was the most tricky one. It required a high performance computing (HPC) cluster of VMs with multiple NVIDIA GPUs designed for compute-intensive, graphics-intensive, and visualization workloads.
This required two things. First, they has to raise money since the amount of computational resources needed was way beyond the $150/Month BizSpark quota they were granted by Microsoft. Second, on the technical side, they had to expand the team and also start setting up a network that connected the VMs they were going to provision. Raising money was very easy since investors were lined up to own a tiny piece of the first flying city. John was able to raise $10M in just two weeks. Given that money they started hiring and added Nora a seasoned data science director heading the analytics department. They also brought Thomas as the director of the computations department which was in charge of building the simulation. Ali’s team was split into two departments: internal kept owning the tools they used for their ITAC managed by a new manager called Jim and a new department was added called external managed by another new manager called Layla in charge building the APIs. PG’s IT was now a 6-department, 30 people team and looked like this:
It was time for execution. Magen set up an executive meeting and started it by this statement: “let’s begin coding our departments”. Except for Ali who started smiling, the rest of the directors and managers started rolling their eyes and looking at each other to make sure they had heard Megan correctly. Nora was the first person to break the silence. “Sorry Megan, did I hear you correctly? you said coding our departments!?”. Megan nodded her head. Ali jumped in. “Let me explain. At PG we define our IT department using code and so we call it ITAC. In an ITAC all the resources, hardware and software are coded, stored in code repository, and deployed using dedicated CI/CD pipelines. Does this make sense?” Layla was the next person to react. “But how would I manage my department? Anybody can go ahead and change the code which means my servers are going to change and that means just chaos! I am not sure if …”. Megan interrupted her and said: “I love your sense of responsibility Layla. Now let me tell you how we have been running PG just like that. Each department owns a project in VSTS that contains all of its resources as code. Only your team and your supervisor, Ali and his supervisor, which is me, have access to that project. You have complete power to assign rights and permissions within your team within your project to define who can update the code and who can approve releasing resources to different environments, as simple as that!”. Jim was really intrigued and said “The is fabulous: we own what we code and we code what we own, right?” Megan said “Absolutely, now I am tasking Ali with re-designing the new ITAC structure. Ali, once you are done with it let’s meet with this group again to review, plan, and execute or shall I say it more accurately review, plan, and code”.