Blog

On the 20th and 21st of this month, Albert Rafel and I attended a 5GPPP workshop in Kista, Stockholm to coordinate the approach to 5G KPIs across a number of EU-funded research projects.

The meeting, convened by Didier Bourse and Anastasius (Tasos) Gavras, had as its stated aim “Boosting the definition of the 5GPPP Performance KPIs at a programme level”. The distinction in the term “programme level” means that the intention was to move the focus from individual projects and their approaches to the KPIs, and to try to arrive at definitions that effectively spanned the large number of projects working in the same area. Of course, in order to achieve that, it was first necessary to hear from each project about its use cases and its approaches to the 5G KPIs. In particular, which KPIs are being addressed by which projects, and how they intend to measure their impact against those KPIs.

Albert and I had several objectives in attending this meeting:

1. Present the Metro-Haul project to show what its objectives are and how it differs from most of the other 5G projects.
2. Describe the Metro-Haul KPIs and how we are attempting to map them to the 5GPPP KPIs.
3. Look for other projects with which we could usefully cooperate, especially in relation to a joint workshop.
4. See whether there are any projects that could work with us on a joint demonstration.

The last of these two objectives represent stretch goals, and the possibility of a joint demo would be a considerable risk because it could easily divert effort from our main project demos that will, in any case, demand a lot of time, planning, and work.

One thing became very clear: not only must Metro-Haul document its KPIs and show how they map to the common 5G KPIs, but the project must find ways to make measurements and compare the behaviour of a network with and without Metro-Haul.

The meeting started with a brief discussion on the existing 5G Architecture White Paper. This paper captures in a rather abstract way the achievements of the phase 1 projects, but the structure, precision, and focus has been criticised by the Commission and by many readers.

Section 7 of this document covers the KPIs and has been extracted to the 5GPPP Phase 2 KPIs document and leads to the “Refined KPIs” in Section 1.2, but these are still not well defined. This work also draws on ITU-R M.2410-0 for radio access KPIs which are more precise, but only apply to the radio access network. The work also captures some responses from a number of projects reflecting their views of KPIs.

If the KPIs are subjective then the whole concept of KGIs is almost impossible to describe and document. Although some projects like to talk about KQIs, the meeting agreed to focus on measurable and quantifiable KPIs.

19 phase 2 5G projects were represented along with 3 phase 3 projects. 3 further phase 2 projects (5GTANGO, MATILDA, and SAT5G) were unable to be present.

As each project made a short presentation, it sounded as though, while most projects were aware of the relevance of some KPIs to the use cases they wanted to address, few were engaged specifically on technical activities designed to improve performance against those KPIs. That is, they were aware of the importance of the KPIs and the effect that they would have on their use cases, but they were not engaged in the development of substantial new technology to improve the performance against the KPIs; rather, they were developing new applications that coordinated and brought together existing technologies in new ways to provide enhanced 5G services.

It also seemed that each project had a slightly different view of the network architecture and software architecture needed to deliver enhanced 5G services. The tangle of architectur, service type, and different technologies are not being untangled with the result that discussion of KPIs (and KQIs) is not shedding much light.

Furthermore, many of the projects are focussed on service delivery that does not spread further than the RAN, or in other cases the immediate aggregation network. This view is somewhat clouded by the different representations of network architecture and the different applications of the terms “aggregation network” and “metro network”.

ONE5G is working to provide end-to-end optimizations for multiple 5G service categories. Recognising that the situation is complicated by the many services and the provision of service chains, the project prefers to look at KQIs and to separate quality assessment into well-defined phases.

Their list of KQIs includes:

• service accessibility (start delay)
• service integrity (bit rate)
• service retainability
• service mobility
• service reliability
• battery life

5GxCAST recognises that broadcast transmissions are key in many 5G use cases such as multimedia and entertainments, connected cars (for infotainment and safety), IoT (for software updates and common control messages), and for public warning and safety systems.

The project maps to a long list of measurable items:

• traffic density
• connection density
• end user experienced data rate
• battery consumption
• user plane e2e delay and latency
• user plane reliability
• end user speed
• minimum expected coverage

5GCAR seeks to develop a vehicle-to-multivehicle system with a range of fine-detail use cases covering lane merge, vehicle see through, automated or remote driving, collaborative map building, etc. This leads to qualitative concerns for cost, power, and security.

With an understanding that “Each use case challenges the network in a different way” the project focus is on stating what levels of KPI are needed to make the functions work.

5G-PHOS is looking to utilise existing photonic technologies to support high-density 5G networks. They are architecting and evaluating 5G broadband networks for dense, ultra-dense, and hotspot area use cases. They aim to leverage 100Gbps and up to 400Gbps over multi-fiber, and high radio access density.

Two main KPIs fall into their area of concern:

• wireless area capacity and more varied service capablities
• facilitating very dense deployments of wireless communication links

But they also recognise “Societal KPIs” such as energy consumption, cost of deployment, and latency and capacity.

IoRL is focussing completely on the wireless part of the network.

The Bluespace project has some similarities to Metro-Haul, but is closer to the edge of the network. Im particular, the “metro network” in Bluespace appears to be just a fiber ring that connect the masts to the aggregation network at the “central office”.

5G-Picture plans to demonstrate converged fronthaul and backhaul services in:

• a smart city environment
• a 5G railway experimental testbed showcasing seamless service provisioning and mobility management in high-speed moving environments
• a stadium with ultra-high user density, supporting media services

The network architecture for this project is very similar to that for Metro-Haul, although the “metro” and “core” networks that they show are simple ring-based topologies.

Monarch has set both technical and economic KPIs and intends to gather stakeholder feedback. The focus is primarily on CRAN (i.e., cloud services local to the radio area). For example, cloud services in a shipping port.

SliceNet use cases cover smartgrid, eHealth, and smart city. They are looking much more at the packet layer than is the case in Metro-Haul, but they have a similar software acrhcitecture and vision. They are building a 10Gbps FPGA packet clssifier, and have discovered that classification delays packets (c. 4 micro seconds) as well as consuming power.

Tasos Gavras was quite keen that we should look at a partnership between SliceNet and Metro-Haul.

NGPaaS has a big architecture picture showing BSS, OSS, etc., right down to device configuration. Their usecases are listed as Telco, 5G mobile, and 5G IoT.

5GCity is targetting the smart city, but its vision and extension to the metro network make its scope similar to that of Metro-Haul.

5GEssence is another CRAN project.

5G-Transformer is looking at a “5G mobile transport platform for verticals” to “bring network slicing into mobile transport networks by provisioning and managing slices tailored to the needs of verticals.”

The project use cases are:

• automotive (pedestrian crossing)
• e-health (heart attack)
• entertainment (on-site fan experience : stadium)
• m(v)no (5G network as a service)
• e-industry (cloud robotics for industrial automation)

All of which have been categorised down to enhanced mobile broadband, mission critical services, and massive IoT.

The three ICT-17 (phase 3) projects present are targeted to provide a framework in which further project proposals can develop their frameworks.

• 5G-EVE is developing an end-to-end validation platform. This will be “An integrated portal for 5G experimentation and validation with inter-working capabilities among trial sites.”

It supports six different verticals

• transport
• tourism
• smart cities
• utilities
• industry 4.0
• media

• 5G-VINNI is a testing framework

• 5Genesis defines an experimentation blueprint to serve as a common reference for implementation of 5G platforms.

The meeting concluded with updates from some of the 5GPPP Working Groups

Architecture WG

• There is a plan to update the overall architecture white paper
• A background question is: can architecture address KPIs?

KPI WG

• Optimisation and measurements need to be compared against base values
• Looking for a champion for each KPI to collate info across projects

Security WG

• Some discussion about the concept of KPIs applied to security. Maybe reword them as objectives.

The Metro-Haul presentation introduced the Metro-Haul objectives and architecture, and listed the project KPIs with which we are familiar. It went on to list specific and updated Metro-Haul KPIs that are distinctive due to the project scope and that can be defined, quantified, and measured.

• Transport & Optical Layer Orientated KPIs
  • P2P connectivity, set-up delay/latency, control layer across optical transport layer
• Control Layer KPIs
  • Service setup delay, network slicing across the optical network, simple service and complex service set-up times, disaggregation to achieve OpEx reductions, and control plane overhead
• Telemetry and monitoring aspects could also have KPIs:
  • Time to detect an anomaly or degradation
  • Amount of data being conveyed to the centralized MDA controller
  • Traffic capture at line rate speed
  • Generation of time series of packets and bytes in a 100 Gbit/s link with a second granularity
  • Sending packet trains at 100 Gbit/s
  • Receiving packet trains at 100 Gbit/s
  • Measuring such packet trains at line rate speed

We also described the next steps which are to:

• Refine the initial project KPIs and define more KPIs specific to the scope of Metro-Haul
• Define measurement methods and used them to determine how well Metro-Haul is doing
• Match-make with some other projects for the purposes of meeting Vertical Service KPI and holding joint workshops or cooperation on demos

Finally, the presentation stressed that it is developing approaches for the metro network, therefore the project does not work directly with the 5G Service KPIs. Instead, the Metro-Haul KPIs focus on delivering solutions that enable the 5G Service KPIs.

The questions we have to answer are about setting relevant KPIs in our scope:

• How do we map 5G Service KPIs to the metro network within the physical constraints of metro networks?
• How would we design/plan/dimension an aggregation network to enable 5G services with an understanding of the scale and demands?
• How should we trade conflicting metro network KPIs?
• What demands do the 5G KPIs and 5G services place on the underlying metro network?
• What additional features are needed in the metro network control and management environment to meet the KPIs?
• What are our innovative solutions that shape the metro network to help deliver the 5G KPIs?