Pages

A new breed of Database

Friday 15 February 2013

Paradigm shift in Database Technology





Gone are the days when organizations heavily used to rely on the their traditional RDBMS for keeping a track of all their transactions and operations. Today is the age of more smarter RDBMS which are made from ground zero for a specific purpose.We will be talking about the OLAP databases in particular, which unconditionally are necessary for every organization playing with large data.
OLAP databases help to derive the meaning of this large data for the organizations. OLAP databases play a major role when it comes to decision making times for organization. The OLAP databases are also called as the lifeline of Decision support systems.



No sooner did the concept of Data warehousing emerged in the IT industry, The Database giants started coming up with their product variants helping to build up a successful Data warehouse.

You have to understand the, The database products which are built with OLTP transactions in mind cannot ever build a good Decision support system. so you have to use a special OLAP database meant for fast analytics and much more.

I will name some Amazing different products in OLAP industry which are taking the horizons to the edge.
1. Vertica
2. Greenplum
3. Sybase
4. Zetaris
5. VoltDB

There are many more in this list you might even have not heard of them, but believe me they exist and make the difference to the Data warehouse Industry

When I personally started to understand Data warehousing Teradata was the first product to be known to me. Its like " Oracle rules Transactional Database and Teradata rules OLAP databases"

I was amazed by many new concepts like shared nothing architecture, MPP architecture, AMP's, Master Node, Slave nodes, Data distribution by has algorithms, and many more exciting new concepts.

While Teradata is still the most liked by the Industry when it comes to building a full proof Data warehousing system. There existing the some really cool products which can change the game and surprise you with their different potentials. Lets discuss some of them

1. Vertica - What is Vertica

Its a pure columnar store MPP database. Yes you read it right Its Columnar
What s columnar ........The data is stored in columnar way rather then row.
So where does it come from ..
It comes from the MIT university from my Favourite Dr Michael Stonebreaker. The project was known to be C-store and it later commercialized into Vertica in year 2005.
C-Store is a read-optimized relational DBMS that contrasts sharply with most current systems, which are write-optimized. Among the many differences in its design are: storage of data by column rather than by row, careful coding and packing of objects into storage including main memory during query processing, storing an overlapping collection of column-oriented projections, rather than the current fare of tables and indexes, a non-traditional implementation of transactions which includes high availability and snapshot isolation for read-only transactions, and the extensive use of bitmap indexes to complement B-tree structures.

2. Greenplum

Greenplum is a RDBMS primarily functions as a data warehouse and utilizes a shared-nothing, massively parallel processing (MPP) architecture. In this architecture, data is partitioned across multiple segment servers, and each segment owns and manages a distinct portion of the overall data; there is no disk-level sharing nor data contention among segments.
While the Greenplum database is built from Most famous and advanced Open source technology PostgreSQL, It is modified to meet the current challenges in the Big Data industry.

3. Sybase IQ

Sybase IQ is a column based, relational database software system used for business intelligence, data warehousing, and data marts. Produced by Sybase Inc., an SAP company, its primary function is to analyze large amounts of data in a low-cost, high-availability environment.The main architectural feature of this version was that for the first time product could support multiple write nodes as well as multiple read nodesThis is a shared everything MPP. (massively parallel processing) architecture.Previously, you could parallelise a query within a node and you could have queries running
in parallel on different nodes but now you can also have a single query distributed across multiple nodes.

4. Zetaris

While this is a hybrid for both OLTP and OLAP, Zetaris is a high performance enterprise-class oltp (on-line transaction processing) and decision support database (scales to over 20tb) with all the expected features of the major proprietary RDBMS vendors. Zetaris is a optimized version of PostgreSQL combined with high standards of flexibility, High availability and many more new features.

5. VoltDB ( OLTP )

This is one an interesting one
Volt DB is an in memory database engine designed by Database researcher named Dr Michael stonebraker, Dr Daniel Abadi. VoltDB is an ACID compliant RDBMS which uses shared nothing architecture.
It again comes from the MIT and Yale university and was well know as H-Store project.
The goal of the H-Store project is to investigate how these architectural and application shifts affect the performance of OLTP databases, and to study what performance benefits would be possible with a complete redesign of OLTP systems in light of these trends.VoltDB is a NewSQL Relational database that supports SQL access from within pre-compiled Java stored Procedure. The unit of transaction is the stored procedure, which is Java interspersed with SQL. VoltDB relies on horizontal partitioning down to the individual hardware thread to scale, k-safety (synchronous replication) to provide high availability, and a combination of continuous snapshots and command logging for durability (crash recovery).
Aren't these amazing technologies conquering the world of  Databases. They don't seem to be ruling, but definitely have made a mark in the recent years and will be changing the way to look at Databases in the upcoming years. They will be a major players in the Big data revolution

Stay tuned for more Technology Facts.....

1 comment: