Efficient broadcast joins in Spark, using Bloom filters
Broadcast joins are a nice way to avoid a shuffle operation in Spark. However, Spark’s collect operation for the broadcast set can introduce memory pressure on the driver. Bloom filters can provide a neat solution to this problem. »
Inside DueDil. Meet Farouk, Data Protection Manager.
Introducing Farouk, DueDil's Data Protection Manager. In this interview, he shares why he enjoys being part of DueDil's engineering team and his journey into the world of tech. »
Reconstructing case information
When you register a business at Companies House, the names are stored in upper case regardless of input. That's a loss of information. We wanted to see if we could reconstruct the "intended" case for these names. »
Inside DueDil. Meet Elia, Product Designer
Meet Elia Kim, Product Designer at DueDil. In this interview, we talk to her about her candidate journey and gather some thoughts on her experience at DueDil so far. »
The Lunch Button
One of the perks of working at DueDil is that lunch gets delivered to the office every Friday. We made a button to alert Slack when it's delivered. »