make_chunks(file_name: str)
mmap.mmap(...)
to create a view of the file as a bytes object, living in virtual memory.mm[start:end]
without loading more data explicitly.cpu_count
to get roughly equal byte sizes.\\n
(newline).
mm
object so that all forked child processes can read from it without copying memory.agg_initializer()
Returns the starting aggregation list: [inf, -inf, 0.0, 0]
which means [min, max, sum, count]
.
defaultdict
needs a way to automatically create a fresh aggregation state for each new city.process_chunk(start, end, queue)
mm[start:end]
of the file.