Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimize error handling after lazy rollovers #108838

Open
nielsbauman opened this issue May 20, 2024 · 1 comment
Open

Optimize error handling after lazy rollovers #108838

nielsbauman opened this issue May 20, 2024 · 1 comment
Labels
:Data Management/Data streams Data streams and their lifecycles :Data Management/Ingest Node Execution or management of Ingest Pipelines including GeoIP >refactoring Team:Data Management Meta label for data/management team

Comments

@nielsbauman
Copy link
Contributor

Currently, we're looping over all the requests in the bulk request for each failed lazy rollover (here, and once #108108 is merged, we'll do it in BulkOperation too). While this is probably around O(n) in practice, it's technically O(n^2), if each document was destined to a different failure store and each rollover failed.

A better alternative would be to collect all the failed rollovers and traverse the documents one time.

@nielsbauman nielsbauman added :Data Management/Ingest Node Execution or management of Ingest Pipelines including GeoIP >refactoring :Data Management/Data streams Data streams and their lifecycles Team:Data Management Meta label for data/management team labels May 20, 2024
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-data-management (Team:Data Management)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Data Management/Data streams Data streams and their lifecycles :Data Management/Ingest Node Execution or management of Ingest Pipelines including GeoIP >refactoring Team:Data Management Meta label for data/management team
Projects
None yet
Development

No branches or pull requests

2 participants