Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Let users configure Cache to use for compiled schemas #2134

Open
stepanho opened this issue Oct 17, 2022 · 4 comments
Open

Let users configure Cache to use for compiled schemas #2134

stepanho opened this issue Oct 17, 2022 · 4 comments

Comments

@stepanho
Copy link

stepanho commented Oct 17, 2022

What version of Ajv you are you using?
8.11.0

What problem do you want to solve?
If compile function called with same by-value, but not by-ref object, it can cause cache overflow and memory leakage.

const getSchema = () => ({ type: 'object' });
const schema1 = getSchema();
const schema2 = getSchema();
console.log('schemas are equal by-ref? ', schema1 === schema2);

// caching is not actually works down here
// Map is growing, nothing cleans it
ajv.compile(schema1);
ajv.compile(schema2);
ajv.compile(getSchema());
...
ajv.compile(getSchema());

What do you think is the correct solution to problem?
Implement WeakMap instead of Map class.

Will you be able to implement it?
I'll attach the pull request.

@stepanho
Copy link
Author

Pull request: #2135

@sempasha
Copy link

sempasha commented Nov 2, 2022

Pull request would be useful to accept. This is good practice to use WeakMap for caching purpose rather then plain Maps. In opposite to plain Map, WeakMap does not hold extra references for its keys, so it does not prevent garbage collector to release memory allocated by keys.

Of course good practice is to avoid schema object duplication (e.g. single object for single schema). But there are some scenarios where cache is not applicable, for example schema could be variable object with short life time.

There are plenty issues caused by Map based cache - #1413 #1879 #1763 #1763.

@epoberezkin
Copy link
Member

Possibly, a better solution to let users configure which cache to use - as it was in the previous versions...

The problem with using WeakMap by default is that it would mask the cases of incorrect schema usages, when the same schema is recompiled, eliminating any benefits of the compilation, while a memory leak helped detecting these issues in most cases.

Also, for cases where schema is dynamic and used only once (or maybe a few times) using compilation is suboptimal, it is more efficient to simply use interpreting validator...

@epoberezkin epoberezkin changed the title Compile cache can cause memory leakage Nov 13, 2022
@sempasha
Copy link

sempasha commented Feb 3, 2023

@epoberezkin do you mean cache option introduced by 0.7.0?

If cache instance is supplied it must have put, get and del methods.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
3 participants