Wayland Benchmark

From frogzie
Jump to navigation Jump to search

1 Purpose

Performance benchmark of Wayland vs X11 display servers

Wayland overview

Wayland is developed as a free and open-source community-driven project with the aim of replacing the X Window System with a modern, secure, simpler windowing system. The Wayland client, XWayland, provides a compatibility layer to seamlessly run legacy X11 applications in Wayland.
Desktop Environments such as Gnome or KDE have now been ported to Wayland.

Test programs

  1. java DrawingBenchmark
  2. glxgears
  3. glmark2

Hardware tested

  1. Laptop ThinkPad X230 with Integrated Intel HD Graphics
  2. Desktop ThinkCentre M92p with Radeon HD 7450 videocard

Additional performance measurement tools used

  • CPU: top   (Solaris mode)
  • Network bandwidth: Gnome System Monitor

2 Scenarios

5 test scenarios were developed

  1. Local run on Gnome / X11
  2. Local run on Gnome / Wayland
  3. Remote X11 connection to Gnome / X11 (export DISPLAY=host:display:screen)
    1. Open Xterm on computer hosting the test program (back-end computer)
    2. Export DISPLAY to computer running the display server (GUI computer)
    3. Run test program from back-end computer with display set to GUI computer
  4. ssh tunnel to display server computer running Gnome / X11
    1. Open Xterm on computer running the display server (GUI computer)
    2. Remote login onto back-end computer hosting the test program (ssh -Y)
    3. Run test program from back-end computer with display onto GUI computer through the ssh tunnel
  5. ssh tunnel to display server computer running Gnome / Wayland
    1. Open Xterm on computer running the display server (GUI computer)
    2. Remote login onto back-end computer hosting the test program (ssh -Y)
    3. Run test program from back-end computer with display onto GUI computer through the ssh tunnel

Note

  • All tests were performed logged into the Gnome desktop;
  • The Wayland display server only supports remote display through ssh tunnels.

3 Performance Benchmark

3.1 Laptop ThinkPad X230 / Integrated Intel HD Graphics

3.1.1 Hardware/Software Configuration

  • Computer running the display server under test: Laptop ThinkPad X230
    • Processor: Intel core i5-3320M @ 2.60 GHz (dual core, 2 threads/core)
    • RAM: 8 GB
    • Integrated Intel HD Graphics
    • Display: 12.5"
    • Screen resolution: 1366x768
    • CentOS 8.1
    • Gnome desktop 3.28-3.32
  • Computer running the remote test program: Laptop ThinkPad X230 (same hardware specifications)
    • Processor: Intel core i5-3320M @ 2.60 GHz (dual core, 2 threads/core)
    • RAM: 8 GB
    • CentOS 7.8
  • Home LAN maximum speed measured at 117 MiB/s

Note: remote X connections are disabled by default. To be able to display a remote X application (-display host:displaynumber), perform the following actions on the display server:

  • Add the instruction "DisallowTCP=false" to the [security] section of /etc/gdm/custom.conf
  • Restart the X server (log out, log back in)
  • Open an Xterm and type "xhost remote_computer_IP"
  • Tell the Redhat/CentOS firewall to allow incoming connections in port 6000 (+ display number (0))
    • firewall-cmd --permanent --zone=public --add-port=6000/tcp
    • firewall-cmd --reload

3.1.2 Java Test

Benchmark source code: DrawingBenchmark.java

DrawingBenchmark.java was run as follows

  1. Local run on CentOS 8 / X11
  2. Local run on CentOS 8 / Wayland
  3. Run from CentOS 7 with DISPLAY exported to CentOS 8 / X11
  4. ssh to CentOS 7 from CentOS 8 / X11
  5. ssh to CentOS 7 from CentOS 8 / Wayland

Detailed results in here

Java Run Display
Server 		Performance Result Display Server
CPU (%) 		Network
(Bandwidth)
1 Local X11 java-x230-local-x11 (not measured) -
2 Wayland java-x230-local-wayland (not measured) -
3 Remote X X11 java-x230-tcp-x11 Xorg/Xwayland: up to 25%
Overall: up to 60% 		up to 115 MiB/s
(100%)
4 Remote ssh X11 java-x230-ssh-x11
5 Wayland java-x230-ssh-wayland

Legend

Overall CPU: computer total CPU usage

Conclusion

  • Performance results are very similar between Wayland and X11, and
  • when X11 is used, performance results are also similar whether the remote display is done by assigning the target address to the environment variable DISPLAY or through an ssh tunnel.

3.1.3 glxgears

glxgears was run as follows

  1. Local run on CentOS 8 / X11
  2. Local run on CentOS 8 / Wayland
  3. Run from CentOS 7 with "-display" directed to CentOS 8 / X11
  4. ssh to CentOS 7 from CentOS 8 / X11
  5. ssh to CentOS 7 from CentOS 8 / Wayland

Results

glxgears Run Display
Server 		Frames/s Display Server
CPU (%) 		Network
(Bandwidth)
1 Local X11 60.1 Xorg: 0.6
Overall: 2.0 		-
2 Wayland 60.0 Xwayland: 0.7
Overall: 2.0 		-
3 Remote X X11 222~223 Xorg: 5.7
Overall: 6~6.5 		90.5 MiB/s
(80%)
4 Remote ssh X11 203~204 Xorg: 2.8~2.9
ssh: 9.8
Overall: ~13 		82~83 MiB/s
(70%)
5 Wayland 207~208 Xwayland: 2.7~2.9
ssh: ~10
Overall: ~13 		85~85.5 MiB/s
(73%)

Legend

Overall CPU: computer total CPU usage

Conclusion

  • Again, performance results are similar between Wayland and X11;
  • However this time there is a noticeable difference between direct remote X connection (-display host:display.screen) and ssh tunnelling. The ssh process required 10% of the overall CPU while the network bandwidth decreased by 10% (default ssh compression). Equally important, the glxgear frame rate dropped by nearly 10 % when using ssh.

Note: for some reason, the remote display frame rate was 3-4 times as fast as the local display one.


3.2 Desktop ThinkCentre M92p / Radeon HD 7450

3.2.1 Hardware/Software Configuration

  • Computer running the display server under test: Desktop ThinkCentre M92p
    • Processor: Intel Core i5-3570 @ 3.40GHz (quad core, 1 thread/core)
    • RAM: 16 GB
    • Videocard: Radeon HD 7450
    • Monitor: 19" ThinkVision
    • Screen resolution: 1280x1024
    • Mageia 7
    • Gnome desktop 3.32
  • Computer running the remote test program: Laptop ThinkPad X230
    • Processor: Intel core i5-3320M @ 2.60 GHz (dual core, 2 threads/core)
    • RAM: 8 GB
    • Mageia 7
  • Home LAN maximum speed at 117 MiB/s

3.2.2 Java Test

Benchmark source code: DrawingBenchmark.java

DrawingBenchmark.java was run as follows

  1. Local run on Desktop / X11
  2. Local run on Desktop / Wayland
  3. Run from Laptop with DISPLAY exported to Desktop / X11
  4. ssh to Laptop from Desktop / X11
  5. ssh to Laptop from Desktop / Wayland

Detailed performance results in here

Java Run Display
Server 		Performance Result Display Server
CPU (%) 		Network
(Bandwidth)
1 Local X11 java-radeon-local-x11 Xorg: up to 17%
Overall: up to 50% 		-
2 Wayland java-radeon-local-wayland Xwayland: up to 26%
Overall: up to 70% 		-
3 Remote X X11 java-radeon-tcp-x11 Xorg: up to 17%
Overall: up to 19% 		up to 80 MiB/s
(70%)
4 Remote ssh X11 java-radeon-ssh-x11 Xorg: up to 18%
ssh: up to 10%
Overall: up to 28% 		up to 110 MiB/s
(95%)
5 Wayland java-radeon-ssh-wayland Xwayland: up to 24%
ssh: up to 15%
Overall: up to 32% 		up to 117 MiB/s
(100%)

Legend

Overall CPU: computer total CPU usage

Summary table

Java Run Display
Server 		Frames per Second
PNG Bitmap Twice Pass 1 PNG Bitmap Twice Pass 2 Plus 2 Sweep Gradient Circles Plus 200 Random Small Circles Plus 320 Long Lines Plus 4000 Random Small Circles
1 Local X11 210.12 214.71 215.61 158.82 85.14 21.00
2 Wayland 602.94 604.44 604.14 486.05 182.13 45.33
3 Remote X X11 213.28 196.14 222.99 138.32 85.01 20.47
4 Remote ssh X11 223.64 194.04 223.88 138.12 89.47 20.40
5 Wayland 347.87 358.43 362.06 330.41 120.82 39.52

Conclusion

Wayland outclassed Xorg in this test, the frame rate being 2.6 times higher during local runs and 1.75 times higher during remote ssh runs (geometric averaging over the sub-tests).

3.2.3 glxgears

glxgears was run as follows

  1. Local run on Desktop / X11
  2. Local run on Desktop / Wayland
  3. Run from Laptop with "-display" directed to Desktop / X11
  4. ssh to Laptop from Desktop / X11
  5. ssh to Laptop from Desktop / Wayland

Results

glxgears Run Display
Server 		Frames/s Display Server
CPU (%) 		Network
(Bandwidth)
1 Local X11 60 Xorg: 0.8
Overall: ~3.2 		-
2 Wayland 60 Xwayland: 0.4
Overall: 2.0~2.9 		-
3 Remote X X11 ~195 Xorg: 5.4
Overall: ~4.1 		~80 MiB/s
(70%)
4 Remote ssh X11 ~191 Xorg: ~2.6
ssh: ~10.7
Overall: ~12.8 		~80 MiB/s
(70%)
5 Wayland ~193 Xwayland: ~2.8
ssh: ~10.8
Overall: ~14 		~80 MiB/s
(70%)

Legend

Overall CPU: computer total CPU usage

Conclusion

  • Performance results are still similar between Wayland and X11, and
  • when X11 is used, the performance difference between remote X connection (-display host:display.screen) and ssh tunnelling appears to be minor.

Note: for some reason, the remote display frame rate was again over 3 times as fast as the local display one.

3.2.4 glmark2

glmark2 was run as follows

  1. Local run on Desktop / X11
  2. Local run on Desktop / Wayland

Note

When run from the ThinkPad X230, glmark2 displayed the following error: "GLX does not support GLX_EXT_swap_control or GLX_MESA_swap_control! Failed to set swap interval. Results may be bounded above by refresh rate." Consequently, all remote test results were discarded.

Results

glmark2 Run Display
Server 		glmark2
Score 		Detailed Result
1 Local X11 356 glmark2-radeon-local-x11
2 Wayland 830 glmark2-radeon-local-wayland

Legend

glmark2 score: the higher the better

Conclusion

Wayland outclassed Xorg in this test with a score more than twice higher.

4 Conclusion

Hardware Test
# 		Test
Program 		Wayland vs X11
(performs better)
ThinkPad X230
Integrated Intel HD Graphics
12.5" Display 1366x768 		1 Java code comparable
2 glxgears comparable
ThinkCentre M92p
Radeon HD 7450
19" ThinkVision 1280x1024 		3 Java code Wayland
4 glxgears comparable
5 glmark2 Wayland
Retrieved from "https://frogzie.duckdns.org/mediawiki/index.php?title=Wayland_Benchmark&oldid=993"