CXYVIP官网源码交易平台_网站源码_商城源码_小程序源码平台-丞旭猿论坛
CXYVIP官网源码交易平台_网站源码_商城源码_小程序源码平台-丞旭猿论坛
CXYVIP官网源码交易平台_网站源码_商城源码_小程序源码平台-丞旭猿论坛

图文并茂丨手把手用C写一个内存池-永久免费的源码丞旭猿

前言

本文从零到一,手把手实现一个内存池。

比较出名的内存池有jemalloc和tcmalloc,这两个都是全局内存池,比较推荐使用tcmalloc。

本专栏知识点是通过零声教育的线上课学习,进行梳理总结写下文章,对c/c++linux课程感兴趣的读者,可以点击链接C/C++Linux服务器开发/后台架构师【零声教育】-学习视频教程-腾讯课堂介绍详细查看课程的服务。

为什么要用内存池

为什么要用内存池?首先,在7 * 24h的服务器中如果不使用内存池,而使用malloc和free,那么就非常容易产生内存碎片,早晚都会申请内存失败;并且在比较复杂的代码或者继承的屎山中,非常容易出现内存泄漏导致mmo的问题。

为了解决这两个问题,内存池就应运而生了。内存池预先分配一大块内存来做一个内存池,业务中的内存分配和释放都由这个内存池来管理,内存池内的内存不足时其内部会自己申请。所以内存碎片的问题就交由内存池的算法来优化,而内存泄漏的问题只需要遵守内存池提供的api,就非常容易避免内存泄漏了。

即使出现了内存泄漏,排查的思路也很清晰。1.检查是不是内存池的问题;2.如果不是内存池的问题,就检查是不是第三方库的内存泄漏。

内存池的使用场景

  1. 全局内存池
  2. 一个连接一个内存池(本文实现这个场景的内存池)

设计一个内存池

总体介绍

由于本文是一个连接一个内存池,所以后续介绍和代码都是以4k为分界线,大于4k的我们认为是大块内存;小于4k的我们认为是小块内存。并且注意这里的4k,并不是严格遵照4096,而是在描述上,用4k比较好描述。

在真正使用内存之前,内存池提前分配一定数量且大小相等的内存块以作备用,当真正被用户调用api分配内存的时候,直接从内存块中获取内存(指小块内存),当内存块不够用了,再有内存池取申请新的内存块。而如果是需要大块内存,则内存池直接申请大块内存再返回给用户。

内存池:就是将这些提前申请的内存块组织管理起来的数据结构,内存池实现原理主要分为分配,回收,扩容三部分。

内存池原理之小块内存:分配=> 内存池预申请一块4k的内存块,这里称为block,即block=4k内存块。当用户向内存池申请内存size小于4k时,内存池从block的空间中划分出去size空间,当再有新申请时,再划分出去。扩容=> 直到block中的剩余空间不足以分配size大小,那么此时内存池会再次申请一块block,再从新的block中划分size空间给用户。回收=> 每一次申请小内存,都会在对应的block中引用计数加1,每一次释放小内存时,都会在block中引用计数减1,只有当引用计数为零的时候,才会回收block使他重新成为空闲空间,以便重复利用空间。这样,内存池避免频繁向内核申请/释放内存,从而提高系统性能。

内存池原理之大块内存:分配=> 因为大块内存是大于4k的,所以内存池不预先申请内存,也就是用户申请的时候,内存池再申请内存,然后返回给用户。扩容=> 大块内存不存在扩容。回收=> 对于大块内存来说,回收就直接free掉即可。

上面理论讲完了,下面来介绍如何管理小块内存和大块内存。

【文章福利】:小编整理了一些个人觉得比较好的学习书籍、视频资料共享在qun文件里面,有需要的可以自行添加哦!~点击832218493加入(需要自取)

小块内存的分配与管理

在创建内存池的时候,会预先申请一块4k的内存,并且在起始处将pool的结构体和node的结构体放进去,

在创建内存池的时候,会预先申请一块4k的内存,并且在起始处将pool的结构体和node的结构体放进去,从last开始一直到end都是空闲内存,中间的区域就用来存储小块内存。每一次mp_malloc,就将last指针后移,直到 e n d − l a s t < s i z e end – last < sizeend−last

初始状态

分配内存

扩容

大块内存的分配与管理

对于大块内存,前面已经说了,用户申请的时候,内存池才申请

  • 申请一块大内存

再申请一块大内存

内存池代码实现

向外提供的api

  • mp_create_pool:创建一个线程池,其核心是创建struct mp_pool_s这个结构体,并申请4k内存,将各个指针指向上文初始状态的图一样。
  • mp_destroy_pool:销毁内存池,遍历小块结构体和大块结构体,进行free释放内存
  • mp_malloc:提供给用户申请内存的api
  • mp_calloc:通过mp_malloc申请内存后置零,相当于calloc
  • mp_free:释放由mp_malloc返回的内存
  • mp_reset_pool:将block的last置为初始状态,销毁所有大块内存
  • monitor_mp_poll:监控内存池状态
structmp_pool_s *mp_create_pool(size_t size);voidmp_destroy_pool(structmp_pool_s *pool);void*mp_malloc(structmp_pool_s *pool, size_t size);void*mp_calloc(structmp_pool_s *pool, size_t size);voidmp_free(structmp_pool_s *pool,void*p);voidmp_reset_pool(structmp_pool_s *pool);voidmonitor_mp_poll(structmp_pool_s *pool,char*tk);

相关结构体的定义

mp_pool_s 就是整个内存池的管理结构,我们做的内存池是一个连接一个内存池,所以对于整个程序而言,内存池对象是有很多个的。

可能读者会有疑问,有了head,为什么还有current,是因为如果一个block剩余空间小于size超过一定次数后,将current指向下一个block,这样就加快内存分配效率,减少遍历次数。

//每4k一block结点structmp_node_s{unsignedchar*end;//块的结尾unsignedchar*last;//使用到哪了structmp_node_s*next;//链表intquote;//引用计数intfailed;//失效次数};structmp_large_s{structmp_large_s*next;//链表intsize;//alloc的大小void*alloc;//大块内存的起始地址};structmp_pool_s{structmp_large_s*large;structmp_node_s*head;structmp_node_s*current;};

内存对齐

访问速度是内存对齐的原因之一,另外一个原因是某些平台(arm)不支持未内存对齐的访问

在4k里面划分内存,那么必然有很多地方是不对齐的,所以这里提供两个内存对齐的函数。那么为什么要内存对齐呢?其一:提高访问速度;其二:某些平台arm不支持未对其的内存访问,会出错。

definemp_align(n, alignment) (((n)+(alignment-1)) & ~(alignment-1))definemp_align_ptr(p, alignment) (void *)((((size_t)p)+(alignment-1)) & ~(alignment-1))

创建与销毁内存池

创建一个线程池,其核心是创建struct mp_pool_s这个结构体,并申请4k内存,将各个指针指向上文初始状态的图一样。   销毁内存池,遍历小块结构体和大块结构体,进行free释放内存。

//创建内存池structmp_pool_s*mp_create_pool(size_t size) {structmp_pool_s*pool;if(size < PAGE_SIZE || size % PAGE_SIZE !=0) {
        size = PAGE_SIZE;
    }//分配4k以上不用malloc,用posix_memalign/*
        int posix_memalign (void **memptr, size_t alignment, size_t size);
     */int ret = posix_memalign((void **) &pool, MP_ALIGNMENT, size);//4K + mp_pool_sif(ret) {returnNULL;
    }
    pool->large = NULL;
    pool->current = pool->head = (unsignedchar*) pool + sizeof(structmp_pool_s);
    pool->head->last = (unsignedchar*) pool + sizeof(structmp_pool_s) + sizeof(structmp_node_s);
    pool->head->end = (unsignedchar*) pool + PAGE_SIZE;
    pool->head->failed =0;returnpool;
}//销毁内存池void mp_destroy_pool(structmp_pool_s*pool) {structmp_large_s*large;for(large = pool->large; large; large = large->next) {if(large->alloc) {
            free(large->alloc);
        }
    }structmp_node_s*cur, *next;
    cur = pool->head->next;while(cur) {
        next = cur->next;
        free(cur);
        cur = next;
    }
    free(pool);
}

提供给用户的内存申请api

申请的内存以size做区分,如果大于4k就分配大块内存,小于4k就去block里面划分。

//分配内存void*mp_malloc(struct mp_pool_s *pool,size_tsize){if(size <=0) {returnNULL;
    }if(size > PAGE_SIZE -sizeof(struct mp_node_s)) {//largereturnmp_malloc_large(pool, size);
    }else{//smallunsignedchar*mem_addr =NULL;structmp_node_s*cur=NULL;cur = pool->current;while(cur) {
            mem_addr = mp_align_ptr(cur->last, MP_ALIGNMENT);if(cur->end - mem_addr >= size) {
                cur->quote++;//引用+1cur->last = mem_addr + size;returnmem_addr;
            }else{
                cur = cur->next;
            }
        }returnmp_malloc_block(pool, size);// open new space}
}void*mp_calloc(struct mp_pool_s *pool,size_tsize){void*mem_addr = mp_malloc(pool, size);if(mem_addr) {memset(mem_addr,0, size);
    }returnmem_addr;
}

小块内存block扩容

所有的block都 e n d − l a s t < s i z e end – last < sizeend−last

//new block 4kvoid *mp_malloc_block(structmp_pool_s*pool, size_t size) {
    unsignedchar*block;
    int ret = posix_memalign((void **) &block, MP_ALIGNMENT, PAGE_SIZE);//4Kif(ret) {returnNULL;
    }structmp_node_s*new_node = (structmp_node_s*) block;
    new_node->end = block + PAGE_SIZE;
    new_node->next = NULL;

    unsignedchar*ret_addr = mp_align_ptr(block + sizeof(structmp_node_s), MP_ALIGNMENT);

    new_node->last = ret_addr + size;
    new_node->quote++;structmp_node_s*current = pool->current;structmp_node_s*cur = NULL;for(cur = current; cur->next; cur = cur->next) {if(cur->failed++ >4) {
            current = cur->next;
        }
    }//now cur = last nodecur->next = new_node;
    pool->current = current;returnret_addr;
}

分配大块内存

//size>4kvoid *mp_malloc_large(structmp_pool_s*pool, size_t size) {
    unsignedchar*big_addr;
    int ret = posix_memalign((void **) &big_addr, MP_ALIGNMENT, size);//sizeif(ret) {returnNULL;
    }structmp_large_s*large;//released struct large resumeint n =0;for(large = pool->large; large; large = large->next) {if(large->alloc == NULL) {
            large->size = size;
            large->alloc = big_addr;returnbig_addr;
        }if(n++ >3) {break;// 为了避免过多的遍历,限制次数}
    }
    large = mp_malloc(pool, sizeof(structmp_large_s));if(large == NULL) {
        free(big_addr);returnNULL;
    }
    large->size = size;
    large->alloc = big_addr;
    large->next = pool->large;
    pool->large = large;returnbig_addr;
}

释放内存

如果是大块内存,找到之后直接释放;如果是小块内存,将引用计数减1,如果引用计数为0则重置last。

//释放内存void mp_free(structmp_pool_s*pool, void *p) {structmp_large_s*large;for(large = pool->large; large; large = large->next) {//大块if(p == large->alloc) {
            free(large->alloc);
            large->size =0;
            large->alloc = NULL;return;
        }
    }//小块 引用-1structmp_node_s*cur = NULL;for(cur = pool->head; cur; cur = cur->next) {//        printf("cur:%p   p:%p   end:%p\n", (unsigned char *) cur, (unsigned char *) p, (unsigned char *) cur->end);if((unsignedchar*) cur <= (unsignedchar*) p && (unsignedchar*) p <= (unsignedchar*) cur->end) {
            cur->quote--;if(cur->quote ==0) {if(cur == pool->head) {
                    pool->head->last = (unsignedchar*) pool + sizeof(structmp_pool_s) + sizeof(structmp_node_s);
                }else{
                    cur->last = (unsignedchar*) cur + sizeof(structmp_node_s);
                }
                cur->failed =0;
                pool->current = pool->head;
            }return;
        }
    }
}

内存池测试

//// Created by 68725 on 2022/7/26.//include 
include 
include 

define PAGE_SIZE4096define MP_ALIGNMENT16define mp_align(n, alignment) (((n)+(alignment-1)) & ~(alignment-1))
define mp_align_ptr(p, alignment) (void *)((((size_t)p)+(alignment-1)) & ~(alignment-1))//每4k一block结点structmp_node_s{
    unsignedchar*end;//块的结尾unsignedchar*last;//使用到哪了structmp_node_s*next;//链表int quote;//引用计数int failed;//失效次数};structmp_large_s{structmp_large_s*next;//链表int size;//alloc的大小void *alloc;//大块内存的起始地址};structmp_pool_s{structmp_large_s*large;structmp_node_s*head;structmp_node_s*current;
};structmp_pool_s*mp_create_pool(size_t size);

void mp_destroy_pool(structmp_pool_s*pool);

void *mp_malloc(structmp_pool_s*pool, size_t size);

void *mp_calloc(structmp_pool_s*pool, size_t size);

void mp_free(structmp_pool_s*pool, void *p);

void mp_reset_pool(structmp_pool_s*pool);

void monitor_mp_poll(structmp_pool_s*pool,char*tk);


void mp_reset_pool(structmp_pool_s*pool) {structmp_node_s*cur;structmp_large_s*large;for(large = pool->large; large; large = large->next) {if(large->alloc) {
            free(large->alloc);
        }
    }

    pool->large = NULL;
    pool->current = pool->head;for(cur = pool->head; cur; cur = cur->next) {
        cur->last = (unsignedchar*) cur + sizeof(structmp_node_s);
        cur->failed =0;
        cur->quote =0;
    }
}//创建内存池structmp_pool_s*mp_create_pool(size_t size) {structmp_pool_s*pool;if(size < PAGE_SIZE || size % PAGE_SIZE !=0) {
        size = PAGE_SIZE;
    }//分配4k以上不用malloc,用posix_memalign/*
        int posix_memalign (void **memptr, size_t alignment, size_t size);
     */int ret = posix_memalign((void **) &pool, MP_ALIGNMENT, size);//4K + mp_pool_sif(ret) {returnNULL;
    }
    pool->large = NULL;
    pool->current = pool->head = (unsignedchar*) pool + sizeof(structmp_pool_s);
    pool->head->last = (unsignedchar*) pool + sizeof(structmp_pool_s) + sizeof(structmp_node_s);
    pool->head->end = (unsignedchar*) pool + PAGE_SIZE;
    pool->head->failed =0;returnpool;
}//销毁内存池void mp_destroy_pool(structmp_pool_s*pool) {structmp_large_s*large;for(large = pool->large; large; large = large->next) {if(large->alloc) {
            free(large->alloc);
        }
    }structmp_node_s*cur, *next;
    cur = pool->head->next;while(cur) {
        next = cur->next;
        free(cur);
        cur = next;
    }
    free(pool);
}//size>4kvoid *mp_malloc_large(structmp_pool_s*pool, size_t size) {
    unsignedchar*big_addr;
    int ret = posix_memalign((void **) &big_addr, MP_ALIGNMENT, size);//sizeif(ret) {returnNULL;
    }structmp_large_s*large;//released struct large resumeint n =0;for(large = pool->large; large; large = large->next) {if(large->alloc == NULL) {
            large->size = size;
            large->alloc = big_addr;returnbig_addr;
        }if(n++ >3) {break;// 为了避免过多的遍历,限制次数}
    }
    large = mp_malloc(pool, sizeof(structmp_large_s));if(large == NULL) {
        free(big_addr);returnNULL;
    }
    large->size = size;
    large->alloc = big_addr;
    large->next = pool->large;
    pool->large = large;returnbig_addr;
}//new block 4kvoid *mp_malloc_block(structmp_pool_s*pool, size_t size) {
    unsignedchar*block;
    int ret = posix_memalign((void **) &block, MP_ALIGNMENT, PAGE_SIZE);//4Kif(ret) {returnNULL;
    }structmp_node_s*new_node = (structmp_node_s*) block;
    new_node->end = block + PAGE_SIZE;
    new_node->next = NULL;

    unsignedchar*ret_addr = mp_align_ptr(block + sizeof(structmp_node_s), MP_ALIGNMENT);

    new_node->last = ret_addr + size;
    new_node->quote++;structmp_node_s*current = pool->current;structmp_node_s*cur = NULL;for(cur = current; cur->next; cur = cur->next) {if(cur->failed++ >4) {
            current = cur->next;
        }
    }//now cur = last nodecur->next = new_node;
    pool->current = current;returnret_addr;
}//分配内存void *mp_malloc(structmp_pool_s*pool, size_t size) {if(size <=0) {returnNULL;
    }if(size > PAGE_SIZE - sizeof(structmp_node_s)) {//largereturnmp_malloc_large(pool, size);
    }else{//smallunsignedchar*mem_addr = NULL;structmp_node_s*cur = NULL;
        cur = pool->current;while(cur) {
            mem_addr = mp_align_ptr(cur->last, MP_ALIGNMENT);if(cur->end - mem_addr >= size) {
                cur->quote++;//引用+1cur->last = mem_addr + size;returnmem_addr;
            }else{
                cur = cur->next;
            }
        }returnmp_malloc_block(pool, size);// open new space}
}

void *mp_calloc(structmp_pool_s*pool, size_t size) {
    void *mem_addr = mp_malloc(pool, size);if(mem_addr) {
        memset(mem_addr,0, size);
    }returnmem_addr;
}//释放内存void mp_free(structmp_pool_s*pool, void *p) {structmp_large_s*large;for(large = pool->large; large; large = large->next) {//大块if(p == large->alloc) {
            free(large->alloc);
            large->size =0;
            large->alloc = NULL;return;
        }
    }//小块 引用-1structmp_node_s*cur = NULL;for(cur = pool->head; cur; cur = cur->next) {//        printf("cur:%p   p:%p   end:%p\n", (unsigned char *) cur, (unsigned char *) p, (unsigned char *) cur->end);if((unsignedchar*) cur <= (unsignedchar*) p && (unsignedchar*) p <= (unsignedchar*) cur->end) {
            cur->quote--;if(cur->quote ==0) {if(cur == pool->head) {
                    pool->head->last = (unsignedchar*) pool + sizeof(structmp_pool_s) + sizeof(structmp_node_s);
                }else{
                    cur->last = (unsignedchar*) cur + sizeof(structmp_node_s);
                }
                cur->failed =0;
                pool->current = pool->head;
            }return;
        }
    }
}

void monitor_mp_poll(structmp_pool_s*pool,char*tk) {
    printf("\r\n\r\n------start monitor poll------%s\r\n\r\n", tk);structmp_node_s*head = NULL;
    int i =0;for(head = pool->head; head; head = head->next) {
        i++;if(pool->current == head) {
            printf("current==>第%d块\n", i);
        }if(i ==1) {
            printf("第%02d块small block  已使用:%4ld  剩余空间:%4ld  引用:%4d  failed:%4d\n", i,
                   (unsignedchar*) head->last - (unsignedchar*) pool,
                   head->end - head->last, head->quote, head->failed);
        }else{
            printf("第%02d块small block  已使用:%4ld  剩余空间:%4ld  引用:%4d  failed:%4d\n", i,
                   (unsignedchar*) head->last - (unsignedchar*) head,
                   head->end - head->last, head->quote, head->failed);
        }
    }structmp_large_s*large;
    i =0;for(large = pool->large; large; large = large->next) {
        i++;if(large->alloc != NULL) {
            printf("第%d块large block  size=%d\n", i, large->size);
        }
    }
    printf("\r\n\r\n------stop monitor poll------\r\n\r\n");
}



int main() {structmp_pool_s*p = mp_create_pool(PAGE_SIZE);
    monitor_mp_poll(p,"create memory pool");if0printf("mp_align(5, %d): %d, mp_align(17, %d): %d\n", MP_ALIGNMENT, mp_align(5, MP_ALIGNMENT), MP_ALIGNMENT,
           mp_align(17, MP_ALIGNMENT));
    printf("mp_align_ptr(p->current, %d): %p, p->current: %p\n", MP_ALIGNMENT, mp_align_ptr(p->current, MP_ALIGNMENT),
           p->current);
endif
    void *mp[30];
    int i;for(i =0; i <30; i++) {
        mp[i] = mp_malloc(p,512);
    }
    monitor_mp_poll(p,"申请512字节30个");for(i =0; i <30; i++) {
        mp_free(p, mp[i]);
    }
    monitor_mp_poll(p,"销毁512字节30个");

    int j;for(i =0; i <50; i++) {char*pp = mp_calloc(p,32);for(j =0; j <32; j++) {if(pp[j]) {
                printf("calloc wrong\n");
                exit(-1);
            }
        }
    }
    monitor_mp_poll(p,"申请32字节50个");for(i =0; i <50; i++) {char*pp = mp_malloc(p,3);
    }
    monitor_mp_poll(p,"申请3字节50个");


    void *pp[10];for(i =0; i <10; i++) {
        pp[i] = mp_malloc(p,5120);
    }
    monitor_mp_poll(p,"申请大内存5120字节10个");for(i =0; i <10; i++) {
        mp_free(p, pp[i]);
    }
    monitor_mp_poll(p,"销毁大内存5120字节10个");

    mp_reset_pool(p);
    monitor_mp_poll(p,"reset pool");for(i =0; i <100; i++) {
        void *s = mp_malloc(p,256);
    }
    monitor_mp_poll(p,"申请256字节100个");

    mp_destroy_pool(p);return0;
}

nginx内存池对比分析

相关结构体定义对比

创建内存池对比

内存申请对比

声明:本文部分素材转载自互联网,如有侵权立即删除 。

© 版权声明
THE END
喜欢就支持一下吧
点赞0赞赏 分享
相关推荐
评论 抢沙发
头像
欢迎您留下宝贵的见解!
提交
头像

昵称

取消
昵称表情代码图片

    暂无评论内容