Migration to google cloud - part 1 - arrival of API V4
At DueDil we are migrating workloads from a managed hardware provider to the Google Cloud Platform (GCP). Services have historically been run on our Mesos cluster, using Marathon for Docker container orchestration »
Ansible on an immutable VM
We use a fair bit of Ansible here at DueDil, and as we try to investigate how to best manage workloads that for one reason or another we don’t want in Kubernetes we've had to do some workarounds that might be helpful for others. »
Efficient broadcast joins in Spark, using Bloom filters
Broadcast joins are a nice way to avoid a shuffle operation in Spark. However, Spark’s collect operation for the broadcast set can introduce memory pressure on the driver. Bloom filters can provide a neat solution to this problem. »
Inside DueDil. Meet Farouk, Data Protection Manager.
Introducing Farouk, DueDil's Data Protection Manager. In this interview, he shares why he enjoys being part of DueDil's engineering team and his journey into the world of tech. »
Reconstructing case information
When you register a business at Companies House, the names are stored in upper case regardless of input. That's a loss of information. We wanted to see if we could reconstruct the "intended" case for these names. »