Cooperative Memory Expansion (COMEX) for
Networked Computing Systems via Remote Direct Memory Access
Overview
Processing big data effectively to ensure high throughput and energy-efficiency with real-time responses has emerged as the new data-intensive computation paradigm, which is expected to grow its importance going forward and also to face mounting challenges associated with “big data” processing. Such a new paradigm urgently calls for innovative solutions to computing system design and implementation.
This project addresses OS kernel support for establishing immense memory collectively across nodes of a networked computing system to hold cold memory pages evicted from the main memory of a host node when executing applications with huge execution footprints well beyond what the host’s main memory can hold, realizing cooperative memory expansion (COMEX). Composed of memory from participating nodes, COMEX permits applications to expand their address space on-demand so that applications with large working sets accelerate their execution runs via utilizing COMEX transparently without application modification while avoiding much slower secondary storages as swap space adopted by native operating systems.
We have implemented COMEX by rectifying process page table, page frame reclaiming, and page fault exception handler subsystems of the Linux kernel 3.10 via OS kernel extensions and a loadable kernel module for network functionalities. Process page table is enhanced to realize the addresses of remote memory page frames for tracking its pages in remote nodes. Rectified page frame reclaiming subsystem allows COMEX to seamlessly move claimed pages to memory of remote nodes during system memory claiming process. Upon a page fault of a remote page referred by faulted page table entry, the rectified page fault exception handler brings the referred page from remote memory back to main memory of the host node, fulfilling process execution's need.
Given its wide adoption by operating systems and networking gear vendors nowadays, the remote direct memory access (RDMA) technology is employed by COMEX for ultra-low latencies in page transfer between nodes whose OS kernel and CPU can be totally bypassed. This project deals with five technical challenges and solutions approaches for the challenges together will constitute the basis of COMEX Handler, able to enhance execution performance and system throughput via better utilizing overall system DRAM memory on-demand. It will also improve the research and educational activities on computer systems and distributed computing on PI’s campus, with its outcomes helping to integrate research and education for enriched teaching, training, and learning experience and to educate quality future scientists critical to the NSF mission.
Expanded Research Activities
In addition, the project has been expanded to investigate into chip multiprocessors (CMPs) and networks on chips (NoCs), which could serve as the building blocks of large-scale distributed systems, where COMEX software under our development can be applied to establish an environment that permits massive in-memory processing. The goals of our expanded research include (1) heightening cache coherence directory storage efficiency for better CMP scalability and (2) developing and assessing Deflection Containment (DeC) in bufferless NoCs for performance improvement and power reduction.
