pandas on spark apply_batch/transform_batch broken? (tl;dr; No – but it isn’t well documented)

Using pypark’s pandas integration via apply_batch and transform_batch is very powerful but lacking documentation can cause hard to trace bugs – hopefully my experience (below)… The post pandas on spark apply_batch/transform_batch broken? (tl;dr; No – but it isn’t well documented) appeared first on Cirrus Minor....

Submit