Pinion — The Load Framework Part-2This post is the 2nd part of the “Pinion — The Load Framework” series. In case you have not read the 1st post, you can read it here.In this post, we are going to cover the following topics.How does Pinion use Delta Lake for SCD operations?Small file problem with Delta Lake and its resolution.Before we dive into the topics of this post, let’s look at the definition of DeltaLake to set the context right.Apache Delta Lake -Apache Delta Lake is an open-source framew...