Distributed tier merge: How LinkedIn tackles stragglers in search index build

Co-authors: Andy Li and Hongbin Wu Indexing plays the key role in modern search engines for fast and accurate information retrieval, and the ability to swiftly build indexes is crucial for LinkedIn to provide up to date information, such as candidates to recruiters, job posts to members, etc. In some instances, such as if a member profile is missing and therefore not searchable, we need to build and deploy a fresh index as part of the solution. To do this, we need an indexing pipeline whose end-...

Submit