A data journey to a dashboard, a newsletter and a bot
Business intelligence engineer Charlotte Emms was initially focused on extracting and analysing data to influence product decisions when she first joined the video collaboration company Seenit. However, her role evolved and she since became the go-to person for anything data-related within the business.
Seenit is a video collaboration tool that eliminates the need for companies to hire film crews to create promotional videos. Instead, employees from within the company or brand enthusiasts from without are entrusted to crowdsource footage and photos. That footage can then be edited in the “studio” within the platform to create promotional media that captures the essence of that brand.
The platform was founded in early 2014 and the minimum viable product was built using Couchbase 2.2, a NoSQL database, deployed on Google Cloud platform. Emms explained that it did not have a query language built into it and so required MapReduce techniques to understand and extract the data in any form.
"With N1QL we could develop features much more rapidly."
In 2015 Seenit introduced N1QL (pronounced nickel) query language and this completely changed the way that Seenit developed features. Emms said: “We could develop features much more rapidly and it also allowed Seenit to hire somebody with a relational database background in Oracle or another relational database, like me, to understand that data.”
As Seenit's only data analyst, Emms had to find a process for dealing with the data requests she was receiving. Once she got a request through, she would write a query in N1QL to extract data from the Couchbase database. She would then use the Python SDK to extract it into a Jupyter Notebook, and finally display it in Plotly. But she said: “This is not a scalable approach for repeatable work.”
She decided to build a dashboard that would display top KPIs for the business in real-time, but once it was deployed it did not get the reception that was anticipated. Emms realised that this was because the KPIs displayed were not relevant to everyone in the business.
"Nobody wants to open email attachments."
Emms then decided to build an email update tool. The email contained information in figures, lists and tables but also contained Excel attachments. “Nobody wants to open attachments. I thought there must be a way that I can display this information in line in an email,” said Emms. She did so and also included a link to the dashboard at the bottom of each email.
To create the email, she used Python in a Jupyter notebook, extracted N1QL queries from Couchbase and “wrangled it around.” “This proof of concept really worked. I showed it to my team. They all thought that it was actually a really good idea, 'It's something we didn't realise that we wanted until you did that'.“
So while the dashboard displayed high-level, monthly and yearly KPIs, the email shows information such as the number of uploads, the uploads with the most likes, and the amount of interaction with the in-app feed – information that was more relevant to the everyday person in the company.
Despite the success of the dashboard and the email, Emms was still concerned that there wasn’t an automated tool that could deliver bespoke analytics, which would be useful for a certain team.
"The customer success team was particularly hungry for data."
“The customer success team was particularly hungry for data because they knew that if they know how their customers are doing, they can tailor their conversations with them much more effectively,” she said. During a meeting the CTO suggested creating a Slackbot, a databot that would deliver bespoke insights on request in the real-time messaging app Slack.
“All you have to do is say a key word followed by whatever you need next. So if you said ‘account stats’ followed by an account ID, it would just generate some grabs for you,” said Emms, who added that it was “dead simple” and really quick to make.
Her advice to data professionals attempting to bring data to the everyday people within a company is to try new things even if they don’t go to plan. Also, not to be intimidated by the technology.
A year ago, she didn’t know what a NoSQL database was but fast forward and she has built automated data programmes on top of a NoSQL database. Neither did she know any Python either but now can code full Python projects.,
Her final piece of advice is to welcome feedback. She said: “You always, always need to gather feedback and keep going with it and keep innovating in your job. It doesn't matter what role you're in, you have space to innovate.”
Charlotte Emms was speaking at Big Data LDN.