When Big Data Meets the World of eCommerce and Lead Generation

When people hear of Computer Science studies, they might think of binary ones and zeros, or nerdy students with high prescription glasses. This may be true in some cases, but most computer science students fall far outside that category and into the everyday norm. However, one thing they do need to have in common if they wish to succeed at their job is the ability to think logically.

Computer Science is a very specific area focused on teaching how to solve problems with logical thinking, rather than making vague assumptions. Its’ a ‘fact driven’ environment, and those facts are fed by data.

This means that people who think in a logical manner and formulate facts from data tend to fit into a business quite easily. Being a Computer Science student (or a student in general!) is a great journey that many of us experience at some stage, and it’s very different from ‘the real’ working environment. Jumping from college straight into a Start-Up Company is a big stretch for a student. You need to gain confidence quickly in how you communicate with other people; both internally within the company, and externally. You will also need to invest heavily on developing the skills that you learned in college, which can be a big challenge. But with a great challenge comes a great opportunity!

Getting to Know the Business: eCommerce and Lead Generation

The very first step is to know which domain the company is operating in – you need a clear understanding of the business goals and objectives before any coding can commence.

In the case of TAMI, our domain is B2B lead generation. Through our technology, we scan the internet and we identify online companies that sell products or services in an online environment; i.e. eCommerce retailers. We then present those websites as sales leads for other businesses who have products or services to supply to them.

Although a ‘sales lead’ might seem self-explanatory, there seems to be some uncertainty around its definition: we talk more about this here, in our article on lead definition. In this industry, we need to understand, not just what a sales lead is, but also what an online sales lead means in the e-commerce world. What’s a good lead, and what’s considered a bad lead? What information is relevant to the customer who would use that sales lead? Clarifying these aspects (and there are many of them) helped in forming a solid understanding of what my job required, and accelerated my integration into our company. Once I had a solid understanding of the business side of things, I could then focus on getting up to speed with the technology environment, development processes, and – in short – working life.

Drowning in Data but Starved for Knowledge

Once I settled down and started working with the team on a data analytics task, I discovered one of the coolest things about being an engineer at TAMI – working with big data. At first, it reminded me of a quote I came across in a business intelligence class: “We’re drowning in data, but starved for knowledge”. In our case, the Internet is a massive source of information and has huge potential for knowledge discovery, but it can also be very overwhelming.

When dealing with such large volumes of data and in so many diverse formats, it’s very difficult to have 100% accuracy at all times – there will always be exceptions because of how messy and cluttered the Internet is. The key to successful data analytics is finding patterns and relations, breaking data into chunks based on those patterns, then sampling and extracting information useful in our business context. This is something that became more like a hobby for me after a while, rather than a job.

And it wasn’t easy to get there – the perfectionist in me demanded 100% accuracy, but analysis can’t go on forever. It has to stop eventually if you want to meet deadlines. We’re constantly trying to improve those last few percentage points by innovating our methods and approaches to achieve more accurate results. And thanks to all of that hard work, our accuracy rate is very high!

Then comes data cleansing – the evil twin of data analytics, where we need to remove false positives, extreme data points, and sometimes even offensive language and terms. This is how I learned and experienced one of the most important things in data analytics for a real working environment – you should never assume anything when it comes to data; it should all be analysed, tested and reviewed before its final presentation.

Summary

Big, established organisations usually enforce years of old development procedures and technical architecture, that a new joiner “simply” has to learn, adapt to, and start working with. However, in a start-up company such procedures are just in their infancy stage, and are perfected over time. This is an excellent opportunity for a graduate – in the space of a few months, they’re gaining incredible practical knowledge; the kind that’s rarely included in a college syllabus.

There’s a LOT to learn, and the only way to find out what you actually know is to keep testing your limits. So it can be a great challenge with steep learning curve at times, but it can also bring massive benefits to you in terms of expanding your knowledge and developing your skills.

On top of that, there’s an opportunity to work directly with great people. And last, but by no means least, there’s the atmosphere. We have 3 different teams in SalesOptimize: sales, marketing, and I.T. But we work so closely together and integrate so well together, that it often feels more like a family to me than a workplace.