There is such a function, it's just not been standardised (and I agree that it is a rather large oversight on the part of the standards committee to have not realised that this is useful information which all implementations of malloc() would already have in some form):
> it also breaks many formerly-well-defined programs that use `malloc_usable_size`
Does it? According to the documentation for malloc_reusable_size, it can be used to find out the actual size of an allocation, but you still have to call realloc before writing to bytes beyond the size passed to malloc.
"This function is intended to only be used for diagnostics
and statistics; writing to the excess memory without first
calling realloc(3) to resize the allocation is not supported."
I have often wished for this. If you can pass a pointer to free() and it frees the previous allocation, it must "know" how big it was, so why is there no way to ask it? You can obviously stash it yourself at malloc() time in a lookup table, but still.
Are there functions available with which I can at run-time query how much OS memory is used, how much handed out in allocations, how many mmap()ed pools are used, and so on?
I find that one of the most important features of a malloc library to debug memory usage.
glibc has these functions (like malloc_info()) -- they are very bugged in that they return wrong results, but after patching them to be correct, they are super useful.
It gets even nastier with systems that care about where memory is allocated. Sometimes you really want to use a particular slab of memory for everything and not just whatever malloc happens to return. You could pass in function pointers or something for malloc/free, but that makes your API even messier.
This sounds like a generally useful thing. It would wager that any non-toy malloc implementation has such a thing. Perhaps it's a good idea to open-source it so we all benefit from this spec-testing, fuzzing and histogramming?
Or, here's a crazy idea: how about we actually allocate the memory when you call malloc(), and if there isn't any, give you an error instead? Programs could check the return code and decide what to do when they run out of memory themselves. Crazy, I know.
Most allocators will be able to pretty efficiently recover the size of the block you are freeing. And you can count on developers not getting the size right, for that to be a common error, and for everyone to cobble together their own, wildly different and probably slower ways of tracking sizes. So it doesn't really help.
malloc/free aren't a great API, but for other reasons (namely, that you want things like multiple arenas, good control over synchronization, decent debugging and introspection, leak-tracking, tagging for figuring out what a block really is when things get smashed, block enumeration, small block pools, placement for cache alignment, and . . . you get the idea).
Do you work on systems that handle this? I'd like to know if any still exist.
In modern systems I'm familiar with, malloc only reports failure on bogus inputs like -1, or address space exhaustion. Your process is likely to be killed before exhausting your address space (think iOS OOM handling, or Linux overcommit), especially on 64 bit. So checking for allocation success just isn't that useful any more.
I was thinking that it’s useful in games or other applications where you don’t want to malloc at arbitrary times (eg in games you might malloc once in startup or on level start and then reuse that memory instead). Many applications with real-time requirements often work similarly (eg high frequency trading), so this would be a fun check to make sure your application is well behaved.
I like the idea of changing pitch by size and also representing frees, but I’m not sure it would be useful to find memory leaks since there’s no real way of matching them up in any non trivial application.
Remember that malloc is a least-common-denominator interface. It needs to work everywhere. In a lot of places, it could be hard to justify the complexity.
Anyway, on glibc systems, you can use malloc_usable_size. On Windows systems (Windows gets the heap very right), you can use the HeapSize function.
There's malloc_usable_size[1], assuming you mean asking the memory allocator what the array size is. But chances are that wouldn't work correctly, because what the amount of memory a malloc calls gives you and the amount you requested are often not the same. Modern memory allocators round up the request size to the nearest "size class".
reply