BulkIndexer memory leak?

[JAndritsch]

I'm observing what appears to be a memory leak when using a BulkIndexer. I noticed that my application's memory was growing rapidly as usage of the BulkIndexer increased. I thought it was maybe my own code in how I was handling the bulk index items, but I commented out the entire thing and was able to replicate the issue by simply triggering a scenario that ran this code successively:

indexer, _ := esutil.NewBulkIndexer(esutil.BulkIndexerConfig{
	Index:  "some-index",
	Client: s.client, // a pointer to elasticsearch.Client
})

ctx := context.Background()

indexer.Close(ctx)

You don't even need to add any items. Just triggering this code multiple times results in a massive increase in memory after a few iterations. My application uses like 8mb of memory after startup. Simply invoking the above code 3-4 times causes that to jump to 160mb. Another time after that and it goes up to 300+mb.

I am using package version v8.17.1.

[JAndritsch]

Update: It seems like memory usage drops down eventually over time (several minutes past the point memory peaks), but the initial spike from ~8mb to 300mb is pretty unexpected. Could there be some internal resource that's eventually timing out and freeing up after some amount of time?

[JAndritsch]

It does seem related to indexer.Close(ctx). If I comment that out and just nil out the ref to the indexer then memory usage doesn't climb unexpectedly. But if I keep indexer.Close(ctx) and still attempt to nil out the indexer ref, I see the unexpected memory usage spike.

So basically...

This leaks:

indexer, _ := esutil.NewBulkIndexer(esutil.BulkIndexerConfig{
	Index:  "some-index",
	Client: s.client, // a pointer to elasticsearch.Client
})

ctx := context.Background()

indexer.Close(ctx)

This does not leak:

indexer, _ := esutil.NewBulkIndexer(esutil.BulkIndexerConfig{
	Index:  "some-index",
	Client: s.client, // a pointer to elasticsearch.Client
})

ctx := context.Background()

//indexer.Close(ctx)

[JAndritsch]

Looks like there's already a PR for this: #701