Maintaining data description is useful feature. There are some ideas, how to implement this.
Read more
[Total: 0 Average: 0/5]
Author: karden
ORM (object-relational mapping) analog for data in data lake
We start saving data in HDFS using avro format. In previous post we have discussed about forward and backward compatibility of avro schemas. How to use this concept in praxis?
Read more
[Total: 0 Average: 0/5]
Running apache zeppelin in k8s cluster and integration with yarn cluster
Useful way for implementing CI/CD pipeline is to pack code as docker and run in k8s cluster. One very practical application for data analytics is notebook based tool apache zeppelin. Every business department requires own configuration for zeppelin. Hence, there is an idea to create docker containers for every department and run in k8s cluster.
Read more
[Total: 0 Average: 0/5]
Book notes – Microservice patterns: with examples in java
Structural minimization of tracking error
The structural tracking error minimization (STEM) approach produces stable tracking portfolios out-of-sample in the crucial investment period. Full version:
Quantitative Finance
[Total: 0 Average: 0/5]
Two infrastructure layers for distributed systems
It looks like, that separation between two infrastructure layers is increasing.
Read more
[Total: 0 Average: 0/5]
Why contracts are important in data intensive applications with microservices
Main purpose of using microservices architecture is to increase velocity of development and reduce system complexity.
Read more
[Total: 0 Average: 0/5]
Protected: Detecting fraud in financial reporting with ML
[Total: 0 Average: 0/5]
Book notes – Kubernetes: Up and Running: Dive into the Future of Infrastructure
Hybrid cloud architecture for data lake applications
Big data technologies nowadays are very mature. Typically you use HDFS, or another distributed file systems, like S3, for storing data, Spark as a processor engine, and YARN as a resource manager. Next steps, wich you probably would like to achieve, are implement CI/CD (continuous integration and delivery) and move workload on demand in cloud.
Read more
[Total: 0 Average: 0/5]