Performance Evaluation of GSM and WCDMA Networks: A Case Study of the University of Ilorin


On this article

Oluwaseun T. Ajayi1 , Samuel O. Onidare2 , Adeseko A. Ayeni2 , Quadri R. Adebowale2 , Saheed O. Yusuf2 and Adeola Ogundele3

1 Department of Electrical and Information Engineering, Covenant University, Canaanland, P.M.B 1023, Ota, Nigeria Department of Telecommunication Science, University of Ilorin, P.M.B 1515, Ilorin, Nigeria 3 Crystalfix Nigeria Limited, Lagos, Nigeria onidare.so@unilorin.edu.ng

Abstract: Mobile networks are evaluated to assess and measure network performance, thereby, improving customer satisfaction and revenue generation. This paper focuses on evaluating the performance of mobile networks, identified as Operator A, Operator B, Operator C, and Operator D, providing voice communication service(s) in the University of Ilorin, Ilorin, Nigeria. A drive test method was used to obtain the key performance indicators (KPIs) for accessibility, retainability, mobility, and service integrity, in comparison with the NCC-defined target for each. The KPI results were aggregated to give the overall network performance for each network operator across the selected drive route – main campus and staff quarters. The Wideband Code Division Multiple Access (WCDMA) network performance in both locations is far behind the Nigerian Communications Commission (NCC) defined KPI target and below the customers' satisfaction, while the Global System for Mobile Communications (GSM) network performance in both locations is permissible. This paper recommends that network operators regularly monitor, optimize and improve on the services provided to subscribers as well as invest in network resources to meet customers' satisfaction as well as increase revenue generation.

Keywords: Drive-test; GSM; KPI; Retainability; WCDMA

1. Introduction

The importance of performance evaluation of mobile networks is inspired by the need to utilize the limited radio resource in the most efficient manner. The continuous increase in demand, for high-quality service, coupled with the future telecommunication network target of providing integrated services with multimedia applications, over low-powered mobile computing devices, has further increased the need for evaluating the performance of mobile networks [1 - 4]. From the users' perspective, the performance of a mobile network is measured in subscribers' satisfaction index, network availability, and efficiency [5 - 8]. However, from technical and regulatory perspectives, network performance evaluation is necessary for assessing and measuring the coverage, capacity, and quality of mobile networks with respect to the service level agreement (SLA) signed with the regulators.

While the operators focus on metrics like the bit error rate (BER), frame erasure rate (FER), and mean opinion score (MOS), the Nigerian Communications Commission (NCC) has identified some measurable key performance indicators (KPIs) to evaluate the success of network operators in Nigeria vis-à-vis subscribers' perception. These KPIs include dropped calls, handover success, blocked calls, interference, coverage, and quality, amongst others. Notably, the KPI results are obtained from the voice and data channels logs on the network [9, 10]. However, to meet the quality of service (QoS) target set by NCC, the following KPI target must be met by all network operators – accessibility (call setup success rate ≥ 98%), retainability (call drop rate ≤ 1%), mobility (handover success rate ≥ 98%) and service integrity [11, 12].

As a national regulatory body vested to establish minimum QoS standards in service delivery for mobile network operators (MNOs) and internet service providers (ISPs), the NCC requires that each network provider reports quarterly, on customer satisfaction in (rural, urban and

Received: September 20th, 2020. Accepted: February 09th, 2021

DOI: 10.15676/ijeei.2021.13.1.5

suburban) Nigeria [13]. This was motivated by the seeming dissatisfaction with the QoS by the rapidly increasing number of information and communications technology (ICT) users across the country [6, 13]. It has, increasingly, become evident and pronounced that the Nigerian telecommunication sector has been plagued with problems, causing frustration among consumers. In recent studies, operators have attributed the poor QoS to the vandalism of network equipment, lack of roads, epileptic power supply, and security breaches in the country. However, mobile network users expect that the increase in technological innovations will foster good speech quality and uninterrupted services [5, 6]. This expectation is based on the perceived increase in the penetration of long-term evolution (LTE) networks in the country and the anticipation of the deployment of 5G networks. The August 2019 to July 2020 report in [13], revealed that for the GSM network, all, but one of the MNOs satisfied the minimum requirement for network accessibility (≥ 98%). This justifies previous statistics by the NCC in May 2016 to April 2017, that amongst the four major MNOs, only one performed outstandingly (satisfying the minimum requirement and beyond) in network accessibility, while one of the MNOs performed poorly in network retainability [14].

The authors in [15] conducted a drive test to evaluate and compare the network performance of four networks (W, X, Y, and Z) in Minna, Nigeria, using Test Mobile System (TEMS). The study validated the claim by NCC that X network had the best network quality in Nigeria. A related study in [5] employed the drive test method with statistical analysis models (chi-square and Fisher's tests) for the performance benchmarking of GSM and Universal Mobile Telecommunications System (UMTS) network operators in Ankara, Turkey. Another study in [4], revealed an analytical survey on the performance evaluation of wireless cellular networks under more realistic assumptions. This study explained that the premature termination of calls (call drop) is possible due to unsuccessful handover of calls when a user moves out of the coverage area of a serving cell, and the target cell has no or limited resources to serve the call connection. A call admission control (CAC) scheme was proposed to reduce the call drop rate [4] [16 - 18]. Results in [5, 15] emphasize the role of regulatory organizations in monitoring and administering wide-scale benchmarking of operators.

The exponential increase in the penetration of broadband communication requires evaluating its performance from an end-user perspective [5, 19]. The authors in [19], conducted a pilot study to benchmark and evaluate the QoS experienced by mobile wireless users in Pakistan by deploying an android application (My Speed Test PK) for mobile customers to measure the performance of the five broadband operators (Ufone, Telenor, Zong, Warid, and Mobilink) in Pakistan. The analysis of the performance obtained by mobile wireless users revealed performance bottlenecks. Improving the mobile broadband internet services (3G and LTE) in Pakistan formed the basis of the pilot study in [20]. The results reinforced the 2015 QoS survey of 3G and 4G for the Pakistan Telecommunication Authority (PTA).

Communication has become the backbone of societal development, with GSM and UMTS being the most used mobile technology for voice and data communications in Nigeria. The adoption of empirical analysis methods [6] and customer perception models [5] in evaluating the degradation of WCDMA networks in Owerri, Nigeria, corroborates the need for the NCC [13, 14] to sanction erring MNOs providing poor services to mobile subscribers. The authors in [6] conducted a survey on the QoS performance of two dominant MNOs in the Owerri metropolis; structured questionnaires were administered to 250 mobile subscribers and results were harvested based on the coverage, accessibility, and mobility KPIs. The authors in [21], mapped out a comparative study of two GSM networks (ZAIN and Globacom) and two CDMA networks (Starcomms and Visafone) by collecting data on calls made in Zaria, Kaduna state. The research considered some KPIs set by NCC such as service accessibility, call setup success rate (CSSR), call setup time, and call completion rate (CCR).

Further study in [22], revealed the performance of GSM networks in Portugal. Results from the study revealed accessibility KPI values of 97.1%, 97.0%, and 96.8% and audio quality KPI values of 56.6%, 50.9%, and 56.9% respectively for the three operators (OPTIMUS, VODAFONE, and TMN) studied. In 2017, P3 Communications carried out a rigorous and

meticulous measurement for the mobile network operators (EE, O2, Vodafone, and Three) in the United Kingdom [23]. The measurements carried out were based on the drive tests and walk tests conducted across 20 large cities and 11 cities in the UK respectively. EE, Vodafone, O2, and Three were rated very good, good, satisfactory, and good respectively.

To effectively cater to subscribers' demand, [24] carried out a research on maximizing coverage, capacity, and quality with guaranteed QoS using a cell cluster optimization approach. Two types of drive tests were conducted to optimize the network – initial drive test and postdrive test. The initial drive test was conducted to serve as a reference level for verifying the performance of the network. The study area was divided into three clusters with each of the clusters having 26 to 40 sites on air. From the drive test analysis, the problem identified across the three GSM clusters were – reduced power output, poor transmission line, increased interference, coverage hole, cross feeder, overshooting, amongst others. After the optimization, the coverage reliability results revealed better coverage performance as well as better quality performance.

While past studies have critically evaluated the performance of GSM and WCDMA/UMTS networks from customers' viewpoint, the study in [25] demystifies the root cause of poor network signal strength and its impact on the QoS provisioning by MNOs. Specifically, the handover procedure and power control were observed in a dense (saturated) network environment using drive test methods. Results from the study emphasized the need for optimal coverage on all the mobile networks, as well as to have the radio frequency (RF) air interface parameters configured to avoid problems of ping-pong handovers, and poor power control, which causes interference and low RSSI among mobile stations (MSs).

In a populated urban area with a literate population such as the University of Ilorin, radio resource availability and reliability is a factor that determines the success of mobile communication. Students and staff, each, complain of poor network accessibility mostly in the academic areas of the University. This incessant complaint which has made many staff and students, each, to subscribe to more than one network operators' service, is an indicator of poor QoS in the University of Ilorin, and by extension most urban centers in Nigeria. On the evidence of this, the networks need to be under continuous monitoring and control, to maintain and improve the performance of the system – an approach that this study seeks to evaluate.

As increase in population and infrastructural development is continuous in any higher institution of learning, the need for improvement in telecommunication services is necessary. This study forms a basis for the regular inspection and optimization of mobile networks in the University of Ilorin, as well as creates a guide for path loss modeling suitable for the campus terrain. Real-time monitoring of radio frequency in the GSM and WCDMA band was conducted. Three drive test locations were selected in the University of Ilorin - University main campus, senior staff quarters, and Jalala junior staff quarters. Possible solutions to improve the network in the selected locations are recommended.

2. Research Method

Table 1. Frequency Definition

BandUplinkDownlinkDuplex
Technology(MHz)FrequencyFrequencyDistance
GSM90089091593596045
DCS1800171017851805188095
UMTS21001920198021102170190

The measurement method employed in this study is the radio frequency drive test – a network test performed to diagnose the root cause of specific radio network problems which are not limited to dropped calls, blocked calls, handover failure, missing neighbor, interference, coverage, and quality [5, 6, 25]. In this study, the measurement is taken for each radio access technology (GSM and WCDMA) and across different bands as shown in Table 1.

A. Measurement Setup

The first phase of the experiment in this study is the data collection phase which requires the use of drive testing tools. These tools are connected to capture and store, as log files, network data in real-time for post-processing and analysis. The tools used in this study include TEMS Investigation, used to capture real-time drive test data, and Test Mobile System (TEMS) used to monitor and analyze mobile networks' performance. Additional tools which constitute the measurement procedure include Sony Ericsson W995, equipped with TEMS software connected to the laptop to capture the network information, activity and data during the period of the drive test, a Power Inverter, and a Global Positioning System (GPS) device for location logging.

B. Drive –Test Route

The University of Ilorin was measured by mapping out routes in the academic area (main campus) and staff residence. The senior and junior staff quarters have access routes linking them, hence, the merger as "Staff Quarters". The drive test was carried out over a total distance of 6.0 kilometers and 7.16 kilometers across the main campus and staff quarters respectively, as shown in Figures 1 - 3.

Figure 1. Main Campus Drive Route

Figure 2. Jalala Junior Staff Quarters Drive Route

Figure 3. Senior Staff Quarters Drive Route

C. Measurement Call Types

In order to obtain the KPI results for GSM and WCDMA networks in the University of Ilorin, two different measurement methods were used – short call and long call, as shown in Table 2.

Table 2. Measurement Call Type

Call TypeDuration (seconds)ParameterMeasured KPI
Short call60Blocked callCall Setup Success
Rate (CSSR)
Long call600Dropped callCall Drop Rate
(CDR)

D. Active Transmitters during Drive-Test

During the drive test, a number of transmitters were on air as shown in Table 3. Specifically, a total of 13, 12, 14 and 8 transmitters were on air for Operator A, Operator B, Operator C and Operator D respectively, during the GSM drive test at the main campus. A total of 15, 8, 6 and 8 transmitters were on air for Operator A, Operator B, Operator C and Operator D respectively, during the GSM drive test at the staff quarters. A total of 21, 12, 17 and 11 transmitters were on air for Operator A, Operator B, Operator C and Operator D respectively, during the WCDMA drive test at the main campus. A total of 14, 8, 8 and 8 transmitters were on air for Operator A, Operator B, Operator C and Operator D respectively, during the WCDMA drive test at the staff quarters.

Table 3. Active Transmitters

GSM (BCCH ARFCN)WCDMA (PSC)
Main Campus
Staff Quarters
Main CampusStaff Quarters
OPERATOR A78, 80, 84,
662, 664, 666,
668, 671, 673,
674, 676, 678,
689
77, 78, 79,
662, 663, 665,
666, 670, 672,
673, 674, 677,
678, 679, 680
0, 1, 2, 21, 26,
27, 36, 63, 125,
205, 206, 207,
217, 218, 219,
291, 301, 307,
325, 357, 420
21, 22, 23, 28,
29, 30, 38, 63,
64, 334, 359,
372, 449, 457
GSM (BCCH ARFCN)WCDM.A (PSC)
Main CampusStaff QuartersMain CampusStaff Quarters
OPERATOR B3, 5, 7, 8, 10,
83, 618, 812,
816, 818, 819,
829
3, 5, 7, 16,
812, 814, 820,
831
37, 73, 75, 77,
115, 253, 282,
291, 294, 399,
409, 413
39, 115, 157,
253, 286, 294,
413, 419
OPERATOR C53, 57, 80, 84,
102, 105, 106,
107, 110, 111,
112, 114, 618,
662
56, 103, 106,
108, 109, 112
6, 66, 71, 101,
117, 125, 142,
163, 188, 259,
291, 299, 307,
330, 432, 478,
480
71, 101, 125,
259, 311, 339,
374, 475
OPERATOR D52, 54, 58,
618, 619, 624,
626, 633
54, 57, 62,
616, 626, 630,
631, 633
125, 129, 133,
291, 299, 301,
307, 334, 403,
449, 450
131, 133, 300,
301, 302, 307,
334, 449

E. Performance Evaluation Metrics

There are four active mobile cellular service providers of the five licensed by the NCC. In this paper, the four active operators, are referred to as Operator A, Operator B, Operator C and Operator D. The operators' network performance is evaluated using two approaches involving two categories of KPI. The first category is the industry defined KPI and the second is the NCC defined KPIs. Table 4 shows the NCC benchmark for each KPI.

  • 1. Industry Defined KPI: Mobile network operators, other than the NCC defined KPIs, use a unique metric network quality index (NQI) in comparing their overall network performance against other competitors in the industry.
    • a. Network Quality Index (NQI): This is a single number that aggregates values of coverage and quality reliability, accessibility, retainability and handover success based on specified weight for each KPI. Accessibility, retainability, speech quality index (SQI), quality reliability, handover success and coverage reliability are allotted weight factors of 30%, 20%, 20%, 15%, 5% and 10% respectively. As shown in equation 1, the NQI is obtained by summing the KPI values with respect to the weight factors.

\[NQI = \sum_{i}^{N} \left( \frac{^{KPI_{i}}/_{4*WF_{i}}}{_{100}} * 100 \right)\] (1)

  • 2. NCC Defined KPI: The relevant KPIs used by NCC to rate the QoS delivered by mobile network operators in Nigeria are:
    • a. Call Setup Success Rate (CSSR): This is the measure of the calls established to the total number of call attempts on a network at a given period of time [5]. Specifically, it is used to evaluate the accessibility of a network, and it is mathematically calculated as:

\[CSSR = \frac{Number of call established}{Total number of call attempt} * 100\] (2)

b. Call Drop Rate (CDR): This metric is used to evaluate the retainability of a network when a call is established on a channel. Call drop rate is mathematically calculated as:

\[CDR = \frac{\text{Number of dropped calls}}{\text{Total number of call established}} * 100\] (3)

Retainability = \[100 - CDR\] (4)

c. Handover Success Rate (HSR): Handover success rate is used to evaluate the performance of network operators relative to user mobility. It is mathematically calculated as:

\[HSR = \frac{Number of successful handover}{Total handover attempt} * 100\] (5)

  • d. Call Setup Time (CST): This is the time taken to establish a call after a successful call attempt.
  • e. Coverage Reliability: This is the measure of the downlink coverage penetration. Voice, SMS and data services depend on the penetration of the downlink coverage. It is mathematically computed as:

\[Coverage = \frac{Samples \ge -85dBm}{Total\ coveage\ sample} * 100\] (6)

f. Network Quality: This is the measure of the signal quality during a call. The RxQuality (GSM) and Ec/No (WCDMA) are measured and used to evaluate a network's quality of reliability. The RxQuality is a parameter which measures the amount of bit errors received by the MS.

Table 4. NCC KPI Definition

KPIsCSSRCDRHSRCSTRx
Level
Rx
Quality
RSCPEc/No
NCC
Target
≥ 98%≤1%≥ 98%≤6sec
-85dBm
≤ 4
-85dBm

-9dBm

3. Results and Discussion

The GSM and WCDMA radio frequency drive test data collected at the main campus and staff quarters of the University of Ilorin, were used to evaluate the performance of the four mobile network operators, identified as Operator A, Operator B, Operator C and Operator D in this study. TEMS discovery device and Microsoft Excel 2013 were used to obtain the KPI results for evaluating both technologies (GSM and WCDMA). Graphs, diagrams and charts were used to reveal the deviation from the NCC defined KPI target for coverage, quality, accessibility, call setup time, retainability and mobility. Also, results from the data revealed which operator had the best performance in these locations. Table 5 and Table 6 show the GSM KPI summary for the main campus and staff quarters respectively, while Table 7 and Table 8 show the WCDMA summary for the main campus and staff quarters respectively.

A. GSM Received Signal Level

At the main campus, Operator A, Operator B, Operator C and Operator D came 1st, 4th, 2nd and 3rd with percentage values of 80.54%, 60.24%, 62.21% and 61.68% respectively. At the staff quarters, Operator A, Operator B, Operator C and Operator D came 2nd, 3rd, 4th and 1st with percentage values of 58.19%, 53.15%, 48.94% and 76.44% respectively. Figure 4 shows the coverage (RxLevel) performance of the operators on the GSM network.

1

Figure 4. (L-R) Received Signal Level in Main Campus and Staff Quarters

B. GSM Call Setup Success Rate

Call setup success rate, is the ratio of established circuit-switched voice call to the total number of call attempts, expressed in percentage. Good downlink coverage penetration and signal quality, foster high call setup success rate, that is, the higher the received signal strength and quality, the higher the number of calls that can be established over a period of time. Channel capacity is also a determinant. However, it is the responsibility of the operators to ensure users of their network have all-time access to network resources (SDCCH/TCH/Cell-DCH) for voice communication.

At the main campus, Operator A, Operator B, Operator C and Operator D came 3rd, 4th, 2nd and 1st with percentage values of 91.67%, 87.67%, 93.33% and 100% respectively. At the staff quarters, Operator A, Operator B, Operator C and Operator D came 2nd, 4th, 3rd and 1st with percentage values of 95.45%, 72.13%, 85.71% and 100% respectively. Figure 5 shows the network accessibility performance of the operators on the GSM network.

6

Figure 5. (L-R) Summary of GSM CSSR in Main Campus and Staff Quarters

C. GSM Call Setup Time

All operators except Operator D, had call setup time within the NCC defined threshold. This implies that Operator D defined a longer call setup time on the network against the NCC defined threshold of ≤ 6 seconds, thereby increasing call setup success rate. Figure 6 shows the performance of each operator with respect to call setup time on the GSM network.

1

Figure 6. (L-R) Summary of GSM CST in Main Campus and Staff Quarters

D. GSM Call Drop Rate

The call drop rate is used to evaluate the retainability of a network when voice traffic is on a channel. The higher the call drop rate, the lower the network retainability and vice versa. At the main campus, Operator A, Operator B, Operator C and Operator D came 4th, 2nd, 1st and 3rd with percentage values of 72.73%, 84.62%, 85.71% and 83.33% respectively in network retainability. At the staff quarters, Operator A, Operator B, Operator C and Operator D came 1st, 4th, 1st and 3rd with percentage values of 100%, 87.50%, 100% and 90.91% respectively in network retainability. Figure 7 shows the network retainability performance of the operators on the GSM network.

5

Figure 7. (L-R) Summary of GSM CDR in Main Campus and Staff Quarters

E. GSM Handover Success Rate

At the main campus, Operator A, Operator B, Operator C and Operator D came 1st, 1st, 4th and 3rd with percentage values of 100%, 100%, 85.71% and 90.00% respectively. At the staff quarters, Operator A, Operator B, Operator C and Operator D came 1st, 1st, 1st and 4th with percentage values of 100%, 100%, 100% and 90.91% respectively. Figure 8 shows the performance of each operator with respect to mobility on the GSM network.

1

Figure 8. (L-R) Summary of GSM HOSR in Main Campus and Staff Quarters

F. GSM Network Quality Index

Table 5. GSM KPI Summary for Main Campus

GSM Main CampusValues (%)Ranks
ABCDABCD
NQI73.3663.9858.8571.111st3rd4th2nd
Coverage
≥ -85 dBm
80.5460.2462.2161.681423
RxQuality ≤ 496.4393.6572.0385.321243
SQI98.8987.9785.4391.561342
Accessibility91.6786.6793.331003421
Retainability72.7384.6285.7183.334213
HSR10010085.7190.001143

Table 6. GSM KPI Summary for Staff Quarters

GSM Staff QuartersValues (%)Ranks
ABCDABCD
NQI69.5040.7667.7887.582nd4th3rd1st
Coverage
≥ -85 dBm
58.1953.1548.9476.442341
RxQuality
≤ 4
75.3281.4679.7598.464231
SQI84.3468.8392.1899.453421
Accessibility95.4572.7385.711002431
Retainability10087.5010090.911413
HSR10010010090.911114
1

Figure 9. (L-R) GSM NQI Summary for Main Campus and Staff Quarters

At the main campus, Operator A, Operator B, Operator C and Operator D came 1st, 3rd, 4th and 2nd with percentage values of 73.36%, 63.98%, 58.85% and 71.11% respectively in network quality index. At the staff quarters, Operator A, Operator B, Operator C and Operator D came 2nd, 4th, 3rd and 1st with percentage values of 69.50%, 40.76%, 67.78% and 87.58% respectively in network quality index. Figure 9 shows the overall performance of each operator using the NQI for the GSM network.

From Table 5 and Table 6, it is apparent that the accessibility, retainability and mobility success of a GSM network hinges on the coverage penetration and signal quality, as shown in Figures 10 – 13.

5

Figure 10. Received Signal Level Penetration in Main Campus

1

Figure 11. Received Signal Quality in Main Campus

3

Figure 12. Received Signal Level Penetration in Staff Quarters

1

Figure 13. Received Signal Quality in Staff Quarters

G. WCDMA Received Signal Code Power (RSCP)

The coverage in WCDMA is defined succinctly by the field strength which is measured by the RSCP. At the main campus, Operator A, Operator B, Operator C and Operator D came 3rd, 4th, 2nd and 1st with percentage values of 39.70%, 20.54%, 54.31% and 64.54% respectively. At the staff quarters, Operator A, Operator B, Operator C and Operator D came 2nd, 4th, 1st and 3rd with percentage values of 32.32%, 10.12%, 39.58% and 32.05% respectively. Figure 14 shows the coverage (RSCP) performance of the operators on the WCDMA network.

5

Figure 14. (L-R) Summary of RSCP in Main Campus and Staff Quarters

H. WCDMA Call Setup Success Rate

At the main campus, Operator A, Operator B, Operator C and Operator D came 1st, 4th, 1st and 1st with percentage values of 100%, 90.00%, 100% and 100% respectively. At the staff quarters, Operator A, Operator B, Operator C and Operator D came 4th, 1st, 1st and 1st with percentage values of 87.50%, 100%, 100% and 100% respectively. Figure 15 shows the network accessibility performance of the operators on the WCDMA network.

3

Figure 15. (L-R) Summary of WCDMA CSSR in Main Campus and Staff Quarters

I. WCDMA Call Drop Rate

At the main campus, Operator A, Operator B, Operator C and Operator D came 3rd, 4th, 2nd and 1st with percentage values of 78.57%, 77.78%, 80.00% and 88.89% respectively in network retainability. At the staff quarters, Operator A, Operator B, Operator C and Operator D came 1st, 3rd, 4th and 2nd with percentage values of 100%, 62.50%, 60.00% and 93.33% respectively in network retainability. Figure 16 shows the network retainability performance of the operators on the WCDMA network.

7

Figure 16. (L-R) Summary of WCDMA CDR in Main Campus and Staff Quarters

J. WCDMA Network Quality Index

At the main campus, Operator A, Operator B, Operator C and Operator D came 3rd, 4th, 2nd and 1st with percentage values of 43.23%, 35.39%, 44.25% and 50.24% respectively in network quality index. At the staff quarters, Operator A, Operator B, Operator C and Operator D came

2nd, 4th, 3rd and 1st with percentage values of 45.30%, 32.68%, 34.78% and 51.36% respectively in network quality index. Figure 17 shows the overall performance of each operator using the NQI for the WCDMA network.

2

Figure 17. (L-R) WCDMA NQI Summary for Main Campus and Staff Quarters

Table 7. WCDMA KPI Summary for Main Campus

WCDMA
Main Campus
Values (%)Ranks
ABCDABCD
NQI43.2335.3944.2550.243rd4th2nd1st
Call Setup Time
≤ 6 sec
10010090.0062.501134
RSCP
≥ -85 dBm
39.7020.5454.3164.543421
Accessibility10090.001001001411
Retainability78.5777.7880.0088.893421
Soft HSR91.5892.0094.0397.304321

Table 8. WCDMA KPI Summary for Staff Quarters

WCDMA Staff QuartersValues (%)Ranks
ABCDABCD
NQI45.3032.6834.7851.362nd4th3rd1st
Call Setup Time
≤ 6 sec
10010080.0035.711134
RSCP ≥ -85 dBm32.3210.1239.5832.052413
Accessibility87.501001001004111
Retainability10062.5060.0093.331342
Soft HSR97.4071.4398.2597.543412

From Table 7 and Table 8, it is evident that the accessibility, retainability and mobility success of a WCDMA network hinges on the coverage penetration (RSCP) and signal quality, as shown in Figures 18 – 21. However, in contrast to the GSM network, the perception of users on the WCDMA network is relatively bad, owing to the poor channel conditions as we traverse the route, thus, making staff and students, each, experience very poor voice quality and more importantly, data dissatisfaction on the WCDMA network.

2

Figure 18. RSCP in Main Campus

4

Figure 19. Ec/No in Main Campus

1

Figure 20. RSCP in Staff Quarters

3

Figure 21. Ec/No in Staff Quarters

Intuitively, the KPI results obtained, show that the GSM network performance in the main campus is better than the performance in the staff quarters. Also, in each location, the WCDMA network performance for Operator A, Operator B, Operator C and Operator D is below the NCC defined KPI target. Problems such as poor RF coverage and quality, dropped call, blocked call and handover failure were prevalent in both locations. At the main campus, the WCDMA network performance is better in the academic areas than at the computer based test (CBT) area

of the institution; contrariwise, at the staff quarters, it is relatively bad for all the operators. Reasons for these problems include – high user equipment transmit power (UE TxPower) and low number of active sites.

4. Conclusion

This study addresses the problems faced by mobile subscribers in the University of Ilorin. Results from the study, show that it is imperative that all network operators regularly monitor, optimize and improve on the services provided to subscribers. Also, the NCC should periodically publicize all operators' KPI results which are obtained by unbiased researchers and sternly criticize/sanction any non-complying operator. This study, however, also serves as a basis for evaluating the preparedness of the mobile network operators in deploying high speed heterogeneous networks (HetNet) – 4G, 5G, in the University of Ilorin.

This research in its entirety, has only addressed the performance of mobile network operators providing circuit-switched voice services in the University of Ilorin. However, for future works related to the scope of this study, researchers should not only evaluate and demystify network problems for voice traffic, but also propose machine learning models to classify radio link problems using parameters such as GPRS link usage, EDGE HTTP uplink/downlink throughput, HSDPA uplink/downlink throughput, latency, amongst others. It is noteworthy that a major challenge militating against teletraffic engineering as well as KPI problem analysis in Nigeria, is the unwillingness of network operators to provide researchers with detailed cell reference data.

5. References

  • [1]. B. M. Kuboye, "Performance Evaluation of Scheduling Algorithms for 4G (LTE)", Communications and Network, vol. 10, no. 4, pp. 152, 2018.
  • [2]. UN News, "Mobile broadband subscriptions on track to hit 4.3 billion in 2017 UN report". [Online]. Available: https://news.un.org/en/story/2017/07/562562-mobile-broadbandsubscriptions-track-hit-43-billion-2017-un-report [Accessed 25/01/2021].
  • [3]. S. U. Abdullahi, "Evaluation of Quality of Service Key Performance Indicators for M-TEL Network of Area VI Kaduna." PhD diss., 2008.
  • [4]. Y. Fang, "Performance Evaluation of Wireless Cellular Networks under more Realistic Assumptions." Wireless Communications and Mobile Computing, vol. 5, no. 8, pp. 867- 885, 2005.
  • [5]. R. Kadioğlu, Y. Dalveren and A. Kara, "Quality of service assessment: A Case Study on Performance Benchmarking of Cellular Network Operators in Turkey". Turkish Journal of Electrical Engineering & Computer Sciences, vol. 23, no. 2, pp.548-559, 2015.
  • [6]. E. M. Onyenwe, O. C. Nosiri, C. Nkwachukwu, and C. K. Agubor, "Performance Evaluation of WCDMA Networks in Selected Geolocations Using Structured Questionnaire and Empirical Analysis Methods". Performance Evaluation, 2018.
  • [7]. C. Konstantinopoulou, K. A. Koutsopoulos, G. L. Lyberopoulos and M. E. Theologou, "Core network planning, optimization and forecasting in GSM/GPRS networks." In Communications and Vehicular Technology, 2000. SCVT-200. Symposium on, pp. 55-61. IEEE, 2000.
  • [8]. T. Halonen, J. Romero and J. Melero, eds. "GSM, GPRS and EDGE Performance: Evolution towards 3G/UMTS". John Wiley & Sons, 2004.
  • [9]. O. J. Ojo, J. J. Popoola, S. A. Oyetunji, Y. O. Olasoji, and M. R. Adu, "Performance Evaluation of a Selected Cellular Mobile Operator in Ibadan Metropolis, Nigeria". Performance Evaluation, vol. 6, no. 5, 2019.
  • [10]. A. R. Mishra, "Fundamentals of Cellular Network Planning and Optimisation," United Kingdom, John Wiley and Sons, 2004
  • [11]. Nigerian Communications Commission, "QoS Metrics & Definitions," [Online]. Available: https://www.ncc.gov.ng/technical-regulation/standards/qos#qos-metrics-definitions. [Accessed 25/01/2021].

  • [12]. Nigerian Communications Commission, "Nigeria consumer Satisfaction survey final report Part 1: Overview for Nigerian communications commission (NCC) consumer Affairs Bureau (CAB) Submitted by NCC consumer Satisfaction survey (NCCCSS) Team: Commonwealth Telecommunications Organisation (CTO)", Fed. Govt. Press, Abuja, 2012
  • [13]. Nigerian Communications Commission, "Quality of Service," [Online]. Available: https://www.ncc.gov.ng/technical-regulation/standards/qos. [Accessed 25/01/2021].
  • [14]. Nigerian Communications Commission, "Quality of Service," [Online]. Available: https://www.ncc.gov.ng/technology/standards/qos#measured-kpisfor-operators. [Accessed 22/12/2017].
  • [15]. A. Ozovehe and A. U. Usman, "Performance analysis of GSM networks in Minna Metropolis of Nigeria." Nigerian Journal of Technology, vol. 34, no. 2, pp. 359-367, 2015.
  • [16]. X. Chen, W. Liu and Y. Fang, "Resource Allocation and Call Admission Control in Mobile Wireless Networks", 2005.
  • [17]. I. Katzela and M. Naghshineh, "Channel assignment schemes for cellular mobile telecommunication systems: A comprehensive survey." IEEE personal communications, vol. 3, no. 3, pp. 10-31, 1996.
  • [18]. S. Tekinay and B. Jabbari, "Handover and channel assignment in mobile cellular networks." IEEE Communications magazine, vol. 29, no. 11, pp. 42-46, 1991.
  • [19]. T. Arshad, M. F. Awan and T. Ahmad, "Performance Evaluation of Mobile Broadband Cellular Networks in Pakistan." In Local Computer Networks Workshops (LCN Workshops), 2016 IEEE 41st Conference on, pp. 104-111, 2016.
  • [20]. M. F. Awan, T. Ahmad, S. Qaisar, N. Feamster and S. Sundaresan, "Measuring broadband access network performance in Pakistan: A comparative study." In Local Computer Networks Conference Workshops (LCN Workshops), 2015 IEEE 40th, pp. 595-602. IEEE, 2015.
  • [21]. C. Kiyea, "Performance analysis of Quality of Service of GSM/CDMA Mobile Networks in Zaria". International Journal of Science and Research (IJSR), 3(10), pp.1247-1253, 2014
  • [22]. ANACOM, "GSM Mobile Networks Quality of Service Survey, Globacombal Study, Mainland, Portugal," ANACOM, 2005. [Online]. Available: https://www.anacom.pt/render.jsp?contentId=344425. [Accessed 12/02/2018].
  • [23]. P3 Connect Mobile Benchmark, "The 2017 Mobile Network Test in the United Kingdom", 2017.
  • [24]. J. Isabona, "Maximising Coverage and Capacity with QOS Guarantee in GSM Network by Means of Cell Cluster Optimization". International Journal of Advanced Research in Physical Science (IJARPS), vol. 1, pp.44-55, 2014.
  • [25]. N. Nkordeh, J. Olatunbosun, I. Bob-Manuel, and O. O. Oni, "Analysis of Mobile Networks Signal Strength for GSM Networks. Analysis of Mobile Networks Signal Strength for GSM Networks". In Proceedings of the World Congress on Engineering and Computer Science, vol. 1, 2016.

OLUWASEUN T. AJAYI received the B.Sc. Telecommunication Science degree from the University of Ilorin, Ilorin, Nigeria in 2018. He is currently working toward the PhD degree in the Department of Electrical and Computer Engineering, Illinois Institute of Technology. His research interests include machine learning in wireless networks, information freshness optimization, and internet of things (IoT) technology.

optimization.

SAMUEL O. ONIDARE received B.Tech degree in Electronic and Electrical Engineering from Ladoke Akintola University of Technology, Ogbomoso, Oyo State, Nigeria, and MSc. in Electrical Engineering with emphasis on Telecommunication from Blekinge Teckniska Hogskola, Karlskrona, Blekinge, Sweden. He is currently pursuing his Ph.D. in communication in the School of Computing and Communication system in Lancaster University, United Kingdom. His research interest includes mobile wireless communication and networks with emphasis on spectrum sharing, green communications, and wireless resource allocation and

AYENI holds a PhD in Electrical Engineering. He has industrial experience in Telecommunication, having worked in NITEL and in ELTEC. He joined academics and rose gradually to become a Professor, having mentored younger scholars and researchers. His research area is wireless communication, radio path profiling, white space detection and radio spectrum sharing. He is a member of IEEE, NSE and he is COREN registered.

ADEBOWALE QUADRI RAMON is a young academic researcher from the prestigious University of Ilorin, Ilorin, Nigeria. He received the B.Sc. Telecommunication Science degree from the University of Ilorin, Nigeria and bagged a Master's degree in Telecommunication Science with focus on wireless and cellular communication, University of Ilorin, graduating with a distinction as the best graduating student. His research interest revolves around channel modelling, machine learning, software defined wireless sensor network, heterogeneous network and Networking. He has published

in various local, National and International journals.

YUSUF SAHEED OLAYINKA is an academic technologist in the department of Telecommunication Science, University of Ilorin. He has a B.Sc. and M.Sc. in Telecommunication from the prestigious University of Ilorin, with a Master's research, focusing on Power Line Communication. His research interests are Channel Coding, Signal penetration, Powerline Network, High Speed Information Networks, Satellite Communication and Optical Fibre Communication. He has published in reputable journals: local, National and International.

ADEOLA OGUNDELE is a graduate of Electronics/Electrical Engineering from Ladoke Akintola University of Technology, Ogbomoso, Nigeria with over 13 years multivendor experience in delivering a wide range of telecoms technology and services. He specializes in 2G/3G/4G/WIMAX/CDMA RF Planning and KPI/Drivetest optimization, transmission/Fiber Network optimization, Engineering Project management and smart technologies. He anchors the Crystalfix Telecoms Training which has produced over 300 RF

Engineers across Africa. He is a member of the Nigerian Society of Engineers and a registered Engineer with COREN.