Tuning spark parameters is not a trivial task. In this short post I will explain how to tune some of the important parameters.
Read more
[Total: 0 Average: 0/5]
Category: DATA PROCESSING
We will discuss here the solutions for Data Processing, like HDFS and Spark.
Book notes – Building Microservices: Designing Fine-Grained Systems
Book notes – Designing Data-Intensive Applications: The Big Ideas Behind Reliable, Scalable, and Maintainable Systems
Why it is a bad idea to stream data back from HDFS into Kafka
Microservices vs service oriented architecture (SOA) and how containers change the rules of the game
Microservices approach gains recently popularity. Some time ago service oriented architecture (SOA) approach was very popular. But what is the difference?
[Total: 0 Average: 0/5]
Schema evolution and backward and forward compatibility for data in data lakes
We have discussed before the format for clean and derived data in data lakes. One of the popular formats for this goal is an avro format. We will talk here why it is needed and how to achieve backward and forward compatibility by designing avro schemas.
Read more
[Total: 0 Average: 0/5]
HBase is next step in your big data technology stack
[Total: 0 Average: 0/5]
Authentication and authorizaton for XMLA Connect and Mondrian
If you would like to turn on basic authentication for mondrian cubes from excel you need to implement steps below.
Read more
[Total: 1 Average: 5/5]
How to implement Kylin dialect for Mondrian
In this post I will explain, how to implement Kylin dialect in Mondrian.
[Total: 1 Average: 4/5]
Improving performance by reading data with Hive for HDFS using subfolders (partitioning)
In our previous article we have discussed the root structure for HDFS. In this article we will discuss next level of the file structure, which will help to improve the speed of reading data.
[Total: 1 Average: 5/5]