• David Woodhouse's avatar
    [JFFS2] Improve read_inode memory usage, v2. · df8e96f3
    David Woodhouse authored
    We originally used to read every node and allocate a jffs2_tmp_dnode_info
    structure for each, before processing them in (reverse) version order
    and discarding the ones which are obsoleted by later nodes.
    
    With huge logfiles, this behaviour caused memory problems. For example, a
    file involved in OLPC trac #1292 has 1822391 nodes, and would cause the XO
    machine to run out of memory during the first stage of read_inode().
    
    Instead of just inserting nodes into a tree in version order as we find
    them, we now put them into a tree in order of their offset within the
    file, which allows us to immediately discard nodes which are completely
    obsoleted.
    
    We don't use a full tree with 'fragments' pointing to the real data
    structure, as we do in the normal fragtree. We sort only on the start
    address, and add an 'overlapped' flag to the tmp_dnode_info to indicate
    that the node in question is (partially) overlapped by another.
    
    When the scan is complete, we start at the end of the file, adding each
    node to a real fragtree as before. Where the node is non-overlapped, we
    just add it (it doesn't matter that it's not the latest version; there is
    no overlap). When the node at the end of the tree _is_ overlapped, we sort
    it and all its overlapping nodes into version order and then add them to
    the fragtree in that order.
    
    This 'early discard' reduces the peak allocation of tmp_dnode_info
    structures from 1.8M to a mere 62872 (3.5%) in the degenerate case
    referenced above.
    
    This version of the patch also correctly rememembers the highest node
    version# seen for an inode when it's scanned.
    Signed-off-by: default avatarDavid Woodhouse <dwmw2@infradead.org>
    df8e96f3
readinode.c 43.1 KB