Projects

B5: Invasive NoCs and Memory Hierarchies for Run-Time Adaptive MPSoCs

Principal Investigators:

Prof. J. Becker, Prof. A. Herkersdorf

Scientific Researchers:

N. Anantharajaiah, L. Masing, S. Rheindt, A. Srivatsa

Abstract

Project B5 investigates and designs the invasible Networks-on-a-Chip, in the following called iNoC.

In the first funding phase, a NoC architecture consisting of hardware modules and protocols was developed and implemented. This iNoC provides the possibility to invade communication resources. Through invasion of communication links, certain Quality-of-Service (QoS) guarantees can be given in terms of upper bounds for bandwidth and latency. Besides this elementary invasion capability, various decentralised and self-optimising mechanisms were investigated. Self-embedding, for instance, enables the decentralised hardware-assisted mapping of communication topologies. Rerouting helps to reduce congestion through remapping of connections and through monitoring of the traffic inside the network adapter links can be invaded autonomously (Auto-GS).

In the second funding phase, the focus of the scientific work shifted from a primarily functional design and implementation of invasion capabilities towards consideration of non-functional properties. This included topics like fault-tolerance, security, power management and dark silicon. The introduction of a lightweight second layer network allows coping with failing routers and can also be used to save energy by manually disabling regular tiles and routers on its path. Furthermore, the distributed memory architecture was improved by enabling applications to efficiently scale beyond tile borders through a novel region-based cache coherence (RBCC) scheme. RBCC provides scalable coherence for large MPSoCs by reducing administrative overheads compared to other, global coherence schemes.

In the third funding phase, Project B5 aims to dynamically adapt the interconnect infrastructure and the memory hierarchy of invasive architectures to optimize non-functional properties like performance or power efficiency. It will be investigated how to align the adaptations with invasive principles and QoS guarantees given by the network. The memory hierarchy will be improved by investigations into data locality optimisations including near-memory computing.

Synopsis

To fully exploit the benefits of invasive computing, the underlying network architecture should be flexible so that any application requirement can be accommodated. Project B5 will investigate techniques for run-time changes in the interconnect architecture by employing several network layers with varying topologies and also investigate the introduction of partial reconfiguration in the iNoC. A key element of research will be the question, how and on which subset of changes the iNoC. can hold and enforce its guarantees during and after any change in the architecture. To manage the adaptability and to optimize network and memory hierarchy special hardware units will be introduced, which employ various techniques to evaluate the current status and initiate run-time optimisations according to different metrics.
Since distributed-shared memory architectures exhibit NUMA properties, we further plan to tackle this data-to-task locality challenge by investigating data and task migration combined with near-memory computing for such hybrid distributed-shared memory system. We envision hardware-assisted migration of data and tasks at system run time, as well as the use of near-memory accelerator modules to improve data-to-task locality. The mutual influence of these techniques with region-based cache coherence will be investigated and a novel concept will be worked out for seamless interaction of these features.

Approach

We plan to investigate hierarchical topologies in both clustered and non-clustered approaches. It will help in increasing fault tolerance by providing redundancy due to the layered structure, on which single layers can also be disabled on demand to allow the enforcement of power ceilings. An example of how we envision clustered and non-clustered iNoC is shown.

To provide a fully flexible architecture, we propose to investigate several techniques that will allow us to change the network infrastructure at varying levels of granularity at run time. Among those techniques we envision the following: (a) Modifying the topology: One way is to change the network and select a topology from a set of predefined topologies. An alternative way is to modify an existing topology. This can be achieved by enabling or disabling links in the network. These unused link and router resources are then re-distributed amongst the layers of the network (b) Switching: A Combination of both packet and circuit switching where we envision a seamless architecture in which both techniques can be joined for the transmission of data. One option to achieve this would be based on additional local ports in each router, in which a packet could switch from the packet-switched network onto a circuit-switched connection that is established within the network. This way, link utilisation in circuit-switched connections can be increased, even while still guaranteeing throughput and latency constraints to the owner (i.e. the initiator) of the circuit. (c) Adaptive routing: In a static topology, adaptive routing will be the focus which, based on traffic load and other parameters, redirects the data to improve performance. In a topology which is changed at run time by, for example enabling and disabling the links, the routing algorithm in the routers is modified to support the changing topology.

To benefit from the newly gained flexibilities in the iNoC, self-management and self-organisation capabilities on either software or hardware level need to be employed. The requirement of any such self-x ability boils down to two parts: Firstly, an understanding of the current status of the system together with the relation between the provided configurations and the target metrics (e.g. requirements regarding non-functional properties of an application). Secondly, a management unit or entity, which can make decisions based on the gathered knowledge and subsequently trigger an adaptation. However, this is not sufficient for efficient execution on invasive architectures, since accommodating competing applications makes run-time knowledge necessary. We will achieve this by employing simple performance monitors, surveying (amongst others) buffer fill levels or virtual channel utilisation. Additionally, probes in the network can be utilised to determine resource availability ahead of time, using network internal communication to help in this process

Our near memory accelerators will implement specific functions in form of hardware that are directly executed on the data in the attached tile local memories and/or the global memory, thereby avoiding latency-costly memory transactions via the interconnect. The execution of NMA functions will occur interleaved with regular memory accesses from processor cores. Thereby, our target is to exploit the available memory bandwidth as much as possible and find a fair share between accesses from NMAs and from the processor cores. For integration into the manycore architecture the NMA modules take part in the coherency protocol of the processor cores including the bus-based invalidation on NMA write operations. For invalidating remote sharers due to the region-based coherency scheme as introduced in Project B5 second phase, an NMA module will therefore tightly interact with the coherency region manager (CRM), which keeps track of sharers within the coherency region.

In general, the goal of data migration is to reduce the number (and also the hop distance) of accesses to remote memories via the NoC within a coherency region. Data should dynamically be migrated to the TLM of the tile where the least latency and energy overhead is generated. To determine what data can be profitably moved and to which target tile, accesses to shared data are continuously monitored on page granularity. Dynamically established iNoC shortcut links or temporarily increased iNoC link bandwidths between source and target tiles will help to reduce migration time and thus make migration more profitable. In this context, an imminent data migration may also be a trigger for a reconfiguration of the iNoC. We will also study the coordination of data-migration with the iNoC self-organisation mechanisms. Dynamically moving data while they are being processed leads to the problem of guaranteeing the atomicity of the migration. We plan to use the diff NMA in tight collaboration with the iRTSS to solve this problem with the following two-step approach: First, the page is copied from its source tile to the target tile while read and write accesses are still allowed at its source tile. However, writes will be recorded by the NMA extension within a page diff until the copy is completed. In the second step, writes are prevented with the help of the iRTSS and the diff is transferred to and applied at the destination tile under the control of the two involved NMA modules. Then the associated page table entry of the migrated page will be updated and the corresponding TLB as well as cache entries will be invalidated by the iRTSS. After that, writes on the page are enabled again. We will study how the concrete interplay with the iRTSS has to occur to guarantee the atomicity of the transfer and also minimise migration costs. It is expected that transferring and applying the diff is much faster than copying the page, thus reducing the time where writes have to be blocked.

Achieved results

A comprehensive summary of the major achievements of the first and second funding phase can be found by accessing Project B5 first phase and Project B5 second phase websites.

Publications