Have you ever written a complex python script that ingests data from multiple sources, processes it, makes some decisions based on the data, and then dumps the output into a database? Or have you developed a formula-heavy spreadsheet that you copy new data into every Monday morning to generate new charts for your CEO? Do you wish you could run some data analysis yourself without having to rely on the every-busy programmers? Turns out, you might just need a data app.
What is an app?
The term “app”, short for “application”, is commonly used to denote a free-standing software that has a very specific set of applications. They are typically built once and deployed with easy-to-use interfaces to a large number of users. This ability of building once, and easily deploying to lots of users makes apps especially popular amongst developers. On the other hand, the accessibility and ease of use make them appealing to end users.
At its core, an app makes it possible for a less technical user to run sophisticated operations, or even a sophisticated user to instituionalize their workflow.
Today, apps are popular on a number of platforms. Mobile apps are by far the most popular category of apps. Similarly, apps have become notably common on platforms like Salesforce and Shopify -- the ZoomInfo app on Salesforce, for example, helps salespeople enrich their leads at the click of a button, and on Shopify the Omnisend app allows e-commerce managers to easily run highly personalized marketing campaigns targeting past customers.
In general, an app on any platform has a few traits:
- While it may have complex operations running under the hood, the interface is usually very easy to use
- It is often tailored to a narrow set of use cases
- The end-user likely does not have the ability to tweak or edit the app
- Once built, it can be easily shared with a large number of users, either within the team or outside
- It enables repeatability and consistency: a process can be reliably executed by any number of end users, any number of times, without introducing human errors generally associated with repetitions
Data Apps/Data Products
Similarly, data apps are a category of apps specifically geared towards helping teams easily manage data-intensive operations. Data operations are all around us, whether in the form of Excel macros, or python scripts that live in Jupyter notebooks, or complex routines maintained by analysts or engineers. Unfortunately, such data operations remain in their silos – they have to be run by the same person on their local computer. This can sometimes lead to chaos when that person unexpectedly has to take a couple weeks off. Additionally, these data operations are often hard to modify and they almost never scale with size.
Most importantly, because of the inherent nature of these operations, business teams (like sales, sales ops, marketing etc) never get to benefit from all the data that exists. They either have to keep poking analytics teams to run the newest analysis for them (which gets old extremely fast!), or data-driven decisions remain a pipe dream.
Components of a data app
At its core, most data apps are built by chaining together one or more of the following components:
- Data Import. This can be an Excel spreadsheet, CSV file, Database/Data warehouse query, CRM (e.g. Salesforce), e-commerce platform (e.g. Shopify)
- Data transformation/computation
- Data preview
- External trigger
- Data export
Data apps built using these data operations are often implemented with a business goal in mind. For example, if the goal is to up-sell next month, you would import and transform your data, and then trigger actions that would recommend the most promising accounts that you should reach out to; the data app could potentially trigger emails itself too.
Given that businesses have different departments and many business goals, they might have tens or hundreds of use-cases for data apps. As data volume and use-cases increase, this process becomes very error prone, and laborious. With these apps, you will be able to automate tasks that don't involve a high degree of human involvement, so data analysts can focus on finding insights and helping business teams to make data-driven decisions.
Why you need to app-ify your data operations
If you touch data or are responsible for making data-driven decisions, you have likely faced one or more of the following situations at work:
- You have built a pretty sophisticated python routine that ingests data from numerous sources, runs some transform operations on it, dumps the data into a spreadsheet, emails the spreadsheet to someone, and records the results in a database. Fortunately, your python routine saves your company a lot of time and money. Unfortunately, the routine runs on your computer in a Jupyter notebook, which means that people are always bothering you to run the newest results. Anytime something needs tweaked, you have to shift your focus from the mission critical tasks on your plate to editing this routine. You wish you could train the non-technical users of the final spreadsheet to run the python routine themselves.
- You have created a formula-rich Excel spreadsheet that performs a series of lookups, pivots and aggregations to ultimately generate a set of graphs that the exec team relies on to make data-driven decisions. You run this first thing every morning. Unfortunately as the scale of the business grows, your spreadsheet slows down considerably. It’s a spreadsheet, and therefore might as well be a house of cards. Anytime you are asked to tweak something, you gingerly edit your spreadsheet, and pray that things won’t completely break down.
- You have built a machine learning model for your sales team to be data-driven in which leads to prioritize. The accuracy is great! However, your sales team is too busy closing deals to learn python in order to run the machine learning model. You host your model for them, and build a user interface to score leads. Unfortunately, after a couple weeks of using the interface, the sales team now wants to send the predicted lead scores to Salesforce -- which means you now have to not focus on your core data competency, but rather on figuring out connections to Salesforce, and productionizing the integration.
In each of these cases, you would have benefited from having data apps. You would have been able to focus on relying on your keen sense of going from data to insights, without having to worry about productionizing operations or building fancy user interfaces. With data apps, you would have built your workflow once, and then empowered your business colleagues to independently make data-driven decisions. Finally, you would have also rested assured knowing that your colleagues were always using data workflows you had developed and signed off on, ensuring accuracy.
Data App: Order Fulfillment
Let’s say that our friend Darryl Philbin is the warehouse manager at Dunder Mifflin, a paper company. It is his job to make sure that all orders placed, either via the sales team or online are shipped on time; he also needs to make sure that those orders that could not be fulfilled are marked as such in a database.
Every morning, Ryan Howard, the intern generates two spreadsheets, one for online orders, and one for salesperson orders. He then emails these spreadsheets to Darryl. Darryl first combines the spreadsheets, checks the details of orders, and compares them against the inventory to see which orders cannot be fulfilled that day. He then aggregates the orders by SKU so he can pull the right inventory from the warehouse onto the shipping station. Next, he looks up each customer’s details in the CRM and pulls their shipping address, and generates a shipping label on Fedex. Finally, he makes a list of unfulfilled orders and sends the list to Ryan, along with a list of shipped orders and shipping tracking numbers. Finally, Ryan updates details for each customer in the CRM. Of course, this process takes a long time for both Ryan and Darryl. Errors are made all too often thanks to the repetitive nature as well.
In a parallel universe (the one with data apps), there's an app for that: every morning, online orders and salesperson orders are pulled and joined, latest inventory details are fetched. Orders are compared against inventory, and a list of out-of-stock orders is generated; orders are aggregated to generate number of units per SKU’s -- this is sent to Darryl; customer details are read from the CRM, and shipping labels automatically generated using a Fedex API endpoint. Customer entries in the CRM are updated instantly, and by the time Ryan is parking his car, he receives a notification that everything ran smoothly.
How to build Data Apps?
Before you create your data apps, you need to answer a fundamental question: do you want to program, update and integrate the apps, or, do you want to use a platform that will deliver all those features out-of-the-box, so you can focus on growing your business and customers?
Whatever way you choose to do it, you should have the following things in mind when choosing your "data app stack":
- Easy integrations with different data sources
- Integrated data preparation
- Sharing options - you'd want your data apps to be accessed by your teammates
- Tracking changes - data teams should be able to maintain provenance with version control
- Audit logs - to monitor the status of
- Cloud based processing that will handle large volumes of data
- Integrated machine learning analytics to take you to the next level
About Intersect Labs
Intersect gets you from data to decisions faster. By using our intuitive visual builder, teams automate repetitive data tasks, trigger actions and ship data apps with prebuilt components. If you are looking for a solution that can help you build your semi-automated data workflows (data apps), let us know.