文章归档

dentry缓存

最近追查某线上服务发现一个问题:频繁的创建&删除目录或者文件,会消耗大量的物理内存。

原因是因为,目录或者文件删除后,dentry并没有立即释放,而是保存在了superblock上的lru链表上。通常只有内核认为内存紧张的时候才会回收此部分内存。

不过由于dentry结构非常小,这种case下内存增长是很慢。

 Active / Total Objects (% used)    : 740781 / 764463 (96.9%)
 Active / Total Slabs (% used)      : 31522 / 31522 (100.0%)
 Active / Total Caches (% used)     : 74 / 101 (73.3%)
 Active / Total Size (% used)       : 144439.65K / 148432.28K (97.3%)
 Minimum / Average / Maximum Object : 0.01K / 0.19K / 8.00K

  OBJS ACTIVE  USE OBJ SIZE  SLABS OBJ/SLAB CACHE SIZE NAME                   
460278 460278 100%    0.19K  21918       21     87672K dentry                 
 91416  81993  89%    0.10K   2344       39      9376K buffer_head            
 30660  29625  96%    0.19K   1460       21      5840K kmalloc-192            
 30080  23012  76%    0.06K    470       64      1880K kmalloc-64             

dentry创建

当用户访问某个路径(/home/work/mysql)时,内核会为这个路径上的每一个节点(home, work, mysql)创建相应的dentry。并且将这些dentry保存到一个hash表里。

dentry删除

当目录或者文件被删除时,如果dentry引用计数为0,就会加入到sb的lru链表上,等待回收。(但也许永远都不释放)

// linux-2.6.32.68/fs/dcache.c
/*
 * dput - release a dentry
 * @dentry: dentry to release 
 *
 * Release a dentry. This will drop the usage count and if 
 * appropriate call the dentry unlink method as well as
 * removing it from the queues and releasing its resources. 
 * If the parent dentries were scheduled for release
 * they too may now get deleted.
 *
 * no dcache lock, please.
 */

void dput(struct dentry *dentry)
{
	if (!dentry)
		return;

repeat:
	if (atomic_read(&dentry->d_count) == 1)
		might_sleep();
	if (!atomic_dec_and_lock(&dentry->d_count, &dcache_lock))
		return;

	spin_lock(&dentry->d_lock);
	if (atomic_read(&dentry->d_count)) {
		spin_unlock(&dentry->d_lock);
		spin_unlock(&dcache_lock);
		return;
	}

	/*
	 * AV: ->d_delete() is _NOT_ allowed to block now.
	 */
	if (dentry->d_op && dentry->d_op->d_delete) {
		if (dentry->d_op->d_delete(dentry))
			goto unhash_it;
	}
	/* Unreachable? Get rid of it */
 	if (d_unhashed(dentry))
		goto kill_it;
  	if (list_empty(&dentry->d_lru)) {
  		dentry->d_flags |= DCACHE_REFERENCED;
                // 这里把dentry放到sb的lru链表里
		dentry_lru_add(dentry);
  	}
 	spin_unlock(&dentry->d_lock);
	spin_unlock(&dcache_lock);
	return;

unhash_it:
	__d_drop(dentry);
kill_it:
	/* if dentry was on the d_lru list delete it from there */
	dentry_lru_del(dentry);
	dentry = d_kill(dentry);
	if (dentry)
		goto repeat;
}

dentry回收

而内存回收有两种方法:

  1. 内核发现内存不足时主动处罚。例如分配不了内存时,达到某种指定的阈值时等等。
  2. 手动触发 sysctl vm.drop_caches。

内核在初始化的时候,注册了一个dcache_shrinker,shrink_dcache_memory的最主要的逻辑时prune_dcache()函数。当用户手动触发内存回收时,这个函数就会被调用

static struct shrinker dcache_shrinker = {
	.shrink = shrink_dcache_memory,
	.seeks = DEFAULT_SEEKS,
};

static void __init dcache_init(void)
{
    int loop;

    /* 
     * A constructor could be added for stable state like the lists,
     * but it is probably not worth it because of the cache nature
     * of the dcache. 
     */
    dentry_cache = KMEM_CACHE(dentry,
        SLAB_RECLAIM_ACCOUNT|SLAB_PANIC|SLAB_MEM_SPREAD);

    // 注册钩子函数,当用户手动触发内存回收时,调用dcache_shrinker
    register_shrinker(&dcache_shrinker);

    /* ... */
}

Leave a Reply

You can use these HTML tags

<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>