Database Selection and Optimisation

Data is the backbone of every organisation today. Storing and querying data efficiently – sometimes in real-time – is one of the major criteria for operational success. And with the changing business needs, the process of selecting a database for your business has evolved as well. Gone are the days when SQL and databases were synonymous. Today, there are many kinds of database platforms to choose from – each serving some specific needs. Our job at Aceso Analytics is to help you select the right kind of database platform and design and optimise it so that you get the most out of your data.

Our Database Selection Process

What will you do with your data? That’s the basic question that needs to be answered in order to select the right database for your business. So, based on this premise, we ask the following questions to assess which database would suit the requirement of a specific organisation.

How Big Will Your Data Be in the Near Term?

The size of data – how many gigabytes your data will consume, how many columns it will have – will impact the choice of database for your organisation. For the majority of projects, SQL-based database platforms – MS SQL Server, MySQL etc. will be more than enough. However, if you are expecting more than, say, 50 TB of data, then we might recommend database engines with parallel processing capabilities (like Redshift).

What Will Be Your Data Type?

Gone are the days when you could shape everything into SQL. In the age of IoT and social media, you’ll get data that can’t fit into any relational database. It is, therefore, necessary to assess from the beginning the kind of data type that you will be working with. For relational data, you can continue using the SQL Server of MySQL. However, in the case of unstructured data, you need NoSQL databases like MongoDB or, better even – Postgres, which can work with both structured and unstructured data.

Will Your Workload Be Read Heavy or Write Heavy?

The choice of database also depends on how your application is going to use it. Will there be predominantly read-heavy tasks, or is the application going to write new data frequently? You might have to leverage caching. As you know, SQL does not support caching query results. In this case, you can use a key-value data store (like Redis) on top of SQL to cache the query results. At Aceso Analytics, we put emphasis on potential read/write frequency as it impacts the database choice.

Can You Separate Your Data Based on the Need for ACID Compliance?

Most businesses deal with critical transactional data for which they need ACID-compliant databases. However, a large part of enterprise data does not demand such strict compliance. If it is possible to segregate data based on the need for ACID compliance, we can use more than one database to achieve low latency and optimal performance. For example, if an organisation handles terabytes of data, we suggest that the organisation use a distributed database for that part of the data, which does not require ACID compliance.

Batch Vs Stream Analytics: Which One Will You Be Using?

Not all companies need real-time analytics. Again some would require a mixture of stream and batch analytics. How an organisation wants to analyse the data impacts the choice of database. For data streaming use cases, instead of traditional databases, a company might benefit from Apache Kafka or AWS Kinesis, which not only supports stream analytics but can also provide features that traditional databases offer.

Lastly, Which Ecosystem Is Your IT Team Familiar with?

At the end of the day, it’s your existing IT team who’ll be managing your organisation’s database. It is, therefore, necessary to assess their proficiency and understand which ecosystem is more familiar to your IT team. For example, an IT team that is familiar with AWS would find it easy to work with Amazon RDS. While it’s not always possible to align the choice of the database with the proficiency of your IT team, at Aceso Analytics, it is our constant endeavour to help you choose a database with a not-so-steep learning curve.

There can be many other questions. But these questions give us a fair idea about which database would be more suitable for your business.

Database Optimisation

Selecting the right kind of database is just one part of the problem. Depending on your use case, your database needs to be optimised so that you get the most out of it. We help you optimise query structure, tweak the structure of your database, avoid loops and improve database indexing in order to keep your db in optimal condition. At Aceso Analytics, we believe that saving space and using the smallest footprint is the soul of database optimisation.

Get In Touch

Choosing the right database is a big decision. The future of your organisation depends on this decision. Contact Aceso Analytics for thorough requirement analysis and a 360-degree assessment of your internal IT maturity, enabling you to select the database that’s best for you.