intermix Blog

Best practices and lessons learned for cloud ETL and data engineering.

Announcing Query Groups – Intelligent Query Classification

1 min READ
August 10th 2018
Query Groups is a powerful feature which intelligently classifies and ranks query workloads on your cluster. Query Groups can answer questions like:my cluster just experienced a sudden increase in latency - which queries are causing this?which queries are consuming the most amount of cluster resourc […]
Paul Lappas Paul Lappas

Crowdsourcing Weather Data With Amazon Redshift

3 min READ
August 6th 2018
Have you ever tried setting up a personal weather station by crowdsourcing weather?  Collecting digital weather data in your backyard or on your rooftop has recently become an easy thing to do. Sharing it is easy also: the Citizen Weather Observer Program (CWOP) will transmit your backyard data to NOAA […]
Lars Kamp Lars Kamp

3 keys to building a stable, reliable, fault-tolerant data pipeline

4 min READ
July 30th 2018
IntroductionWhen you’re building or improving your data pipeline, batch data processing seems simple. Pull data from a source, apply some business logic to it, and load it for later use. When done well, automating these jobs is a huge win. It saves time and empowers decision-makers with fresh and accura […]
Cody Hanson Cody Hanson

Improve Amazon Redshift COPY performance:  Don’t ANALYZE on every COPY

2 min READ
July 24th 2018
IntroductionOne of the core challenges of using any data warehouse is the process of moving data to a place where the data can be queried. Amazon Redshift COPY command provides two methods to access data:1- copy data into Redshift local storage by using the COPY command2- use Amazon Redshift […]
Paul Lappas Paul Lappas

Zero Downtime Elasticsearch Migrations

6 min READ
July 12th 2018
IntroductionAt, Elasticsearch is a final destination for data that is processed through our data pipeline. Data gets loaded from Amazon Redshift into Elasticsearch via an indexing job. Elasticsearch data then gets served to the dashboard to data engineers, giving them a view […]
Stefan Gromoll Stefan Gromoll
1 7 8 9 10 11