I would like to share my Pair programming exercise with you!

I believe that building a product along with a team is more powerful than building it alone. I had some chances to participate in pair programming in my experience.

1. [I was the driver] While I started learning ATG (Oracle commerce), there was one workstation provided and two people were allocated. We learned ATG as fast as possible in one month which was the most efficient and productive learning experience I had.

2. [I was the navigator] The second experience with my Technical Lead who was very strong in…

The role of OLAP data warehouse is very important in today’s Big data analytics era.

But, comparing to last decade, the usage of volume of data in analytics is increasing. The AI & ML are very hungry and it needs tera bytes of data with complex query.

The modern BI tools like Qliksense, PowerBI, Tableau can talk to the large volume data warehouse. But if we think about low latency in very large volume of data, something needs to be changed in the architecture.

Change in Architecture

The first generation platforms architecture can be used in the normal big data analytics.

Managing MonoRepo having its own positives and negatives. But most of the products are moving towards MonoRepo architecture.

I am having some example to maintain a MonoRepo from MultiRepo architecture.

I am using Dagster for data pipeline. I am not sure whether its fit on this box...

One of the magic of the Clickhouse analytics database is that query/analyse the data directly from a CSV file

Create a basic View

First we’ve to create a basic view of a file. For that we need to have the data types of each column.

CREATE VIEW sales_view AS
   SELECT * FROM FILE(
          '/data/sales.csv',
          'CSV',
          '`order_id` Int64, `sales_amt` Float64);

This will create a view in database

Query the view

After this it will act as a normal view. You can query the CSV file through this table view.

SELECT sum(`sales_amt`) from sales_view;

The magic

The magic here is, the data not loaded into the database storage, instead the data…

a long waiting feature :) thanks @Dagster

This is important to every Software developer to get the appreciation from their Management or Client side to get boost on their Career.

Getting appreciation in front of their co-workers in an occasion, is more inspiration than getting monetary benefit in the Bank.

But, as an Engineer, if I don’t have it from the society, I’ve to create it and dedicate to myself. By doing this, I am appreciating my heart and brain which are functioning for that growth.

Inspiration

From last ten days, the people from Born posting their appreciation certificates on LinkedIn. …

The year 2018 was the wonderful year for me. I explored lot of tools both in Machine learning and Programming aspects.

1. CDAP

The data integration, pipeline and workflow management system which helped me to create the automation system and reduced 80% of my manual work. Also used Python Flask as the web service to run my other colleagues code from CDAP HTTP sync process

2. Python Flask

Flask is the powerful Python tool and helped me to build the REST API in 5minutes. Created the 2D graph tool using Flask as the backend service with Pandas data frames

3. Adobe XD

Of-course, the UI / UX is…