Skip to content

Commit caf8391

Browse files
rfj82982rfj82982
committed
Merge branch 'main' of github.com:2decomp-fft/2decomp-fft.github.io into main
2 parents b919c53 + fe0cd96 commit caf8391

File tree

64 files changed

+479
-5
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

64 files changed

+479
-5
lines changed

‎docs/source/pages/api_domain.rst‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,9 @@
33
===========================
44

55
This page explains the key public interfaces of the 2D decomposition library. After reading this section, users should be able to easily build applications using this domain decomposition strategy.
6-
The library interface is designed to be very simple. One can refer to the sample applications for a quick start.
6+
The library interface is designed to be very simple. One can refer to the
7+
`example applications <https://github.com/2decomp-fft/2decomp-fft/tree/main/examples>`_
8+
for a quick start.
79

810
The 2D Pencil Decomposition API is defined in three Fortran module which should be used by applications as:
911

‎docs/source/pages/benchmarks.rst‎

Lines changed: 69 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,72 @@
1+
.. role:: raw-html(raw)
2+
:format: html
3+
4+
.. |br| raw:: html
5+
6+
<br/>
7+
18
==========
29
Benchmarks
310
==========
11+
12+
This page report the result of some scalability tests which are available in the under
13+
`example <https://github.com/2decomp-fft/2decomp-fft/tree/main/examples>`_
14+
The test being performed are the following:
15+
16+
#. `Transpose of a real <https://github.com/2decomp-fft/2decomp-fft/blob/main/examples/test2d/timing2d_real.f90>`_
17+
3D array.
18+
19+
#. `Transpose of a complex <https://github.com/2decomp-fft/2decomp-fft/blob/main/examples/test2d/timing2d_complex.f90>`_ 3D array.
20+
21+
#. 3D FFT trasform
22+
`(fft_r2c_x) <https://github.com/2decomp-fft/2decomp-fft/blob/main/examples/fft_physical_x/fft_r2c_x.f90>`_
23+
of a **real** 3D array starting from `X` physical direction.
24+
The trasform has both forward and backward to retrieve the inout array.
25+
26+
#. 3D FFT trasform
27+
`(fft_c2c_x) <https://github.com/2decomp-fft/2decomp-fft/blob/main/examples/fft_physical_x/fft_c2c_x.f90>`_
28+
of a **complex** 3D array starting from `X` physical direction.
29+
The trasform has both forward and backward to retrieve the inout array.
30+
31+
#. 3D FFT trasform
32+
`(fft_r2c_z) <https://github.com/2decomp-fft/2decomp-fft/blob/main/examples/fft_physical_z/fft_r2c_z.f90>`_
33+
of a **real** 3D array starting from `Z` physical direction.
34+
The trasform has both forward and backward to retrieve the inout array.
35+
36+
#. 3D_FFT_trasform
37+
`(fft_c2c_z) <https://github.com/2decomp-fft/2decomp-fft/blob/main/examples/fft_physical_z/fft_c2c_z.f90>`_
38+
of a **complex** 3D array starting from `Z` physical direction.
39+
The trasform has both forward and backward to retrieve the inout array.
40+
41+
All timing are collected averaging 50 repetitions of the test with the 0 iteration being discarded.
42+
Two resolutions have been tested:
43+
44+
* ``NX=NY=NZ=512`` which corresponds to rougly 130 million points.
45+
46+
* ``NX=NY=NZ=1024`` which corresponds to rougly 1 billion points.
47+
48+
A **2D** label for the results indicates a 2D (i.e. pencils) decomposition using the optimal automatic configuration
49+
(that generally corresponds to the closest decomposition to ``NR=NC``).
50+
A **1D** label for the results indicates a 1D (i.e. slabs) decomposition. With 2DECOMP&FFT this is obatained
51+
forcing one of the two decomposition direction to 1. If ``N_ROW=1`` an initial ``Z`` slabs
52+
(i.e. local memory data are in the ``XY`` plane) is obatained,
53+
conversely ``N_COL=1`` start from a ``X`` slabs configuration
54+
(i.e. local memmory data are in the ``YZ`` plane).
55+
Generally only one set of slabs data are plotted since performances are relatively similar.
56+
57+
58+
The library has been benchmark on the following systems:
59+
60+
* The UK National Supercomputer service `Archer2 <https://www.archer2.ac.uk>`_
61+
62+
* The GPU partition of the EPCC `Cirrus <https://github.com/2decomp-fft/2decomp-fft/tree/main/examples>`_
63+
service
64+
65+
66+
.. toctree::
67+
:maxdepth: 1
68+
:caption: Here the detailed list of the results:
69+
70+
benchmarks_archer2.rst
71+
benchmarks_cirrus.rst
72+

‎docs/source/pages/benchmarks_archer2.rst‎

Lines changed: 164 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,164 @@
1+
.. role:: raw-html(raw)
2+
:format: html
3+
4+
.. |br| raw:: html
5+
6+
<br/>
7+
8+
.. _benchmark-archer2:
9+
10+
========================
11+
Results for CPU: Archer2
12+
========================
13+
14+
Archer2 is the UK National Supercomputer service capable of 28 Pflops/s at peak performance.
15+
The systems has 5,860 compute nodes, each with dual AMD EPYCTM 7742 64-core processors at 2.25GHz,
16+
giving 750,080 cores in total.
17+
18+
Resuls are listed below
19+
20+
* Transpose real 3D array
21+
22+
* Resolution ``NX=NY=NZ=512`` |br| |CPU_0512_TrReal_Scal| |CPU_0512_TrReal_SpeedUp|
23+
24+
* Resolution ``NX=NY=NZ=1024`` |br| |CPU_1024_TrReal_Scal| |CPU_1024_TrReal_SpeedUp|
25+
26+
* Transpose complex 3D array
27+
28+
* Resolution ``NX=NY=NZ=512`` |br| |CPU_0512_TrClx_Scal| |CPU_0512_TrClx_SpeedUp|
29+
30+
* Resolution ``NX=NY=NZ=1024`` |br| |CPU_1024_TrClx_Scal| |CPU_1024_TrClx_SpeedUp|
31+
32+
* FFT transform of a 3D real array starting from ``X`` physical direction
33+
34+
* Resolution ``NX=NY=NZ=512`` |br| |CPU_0512_R2CX_Scal| |CPU_0512_R2CX_SpeedUp|
35+
36+
* Resolution ``NX=NY=NZ=1024`` |br| |CPU_1024_R2CX_Scal| |CPU_1024_R2CX_SpeedUp|
37+
38+
* FFT transform of a 3D complex array starting from ``X`` physical direction
39+
40+
* Resolution ``NX=NY=NZ=512`` |br| |CPU_0512_C2CX_Scal| |CPU_0512_C2CX_SpeedUp|
41+
42+
* Resolution ``NX=NY=NZ=1024`` |br| |CPU_1024_C2CX_Scal| |CPU_1024_C2CX_SpeedUp|
43+
44+
* FFT transform of a 3D real array starting from ``Z`` physical direction
45+
46+
* Resolution ``NX=NY=NZ=512`` |br| |CPU_0512_R2CZ_Scal| |CPU_0512_R2CZ_SpeedUp|
47+
48+
* Resolution ``NX=NY=NZ=1024`` |br| |CPU_1024_R2CZ_Scal| |CPU_1024_R2CZ_SpeedUp|
49+
50+
* FFT transform of a 3D complex array starting from ``Z`` physical direction
51+
52+
* Resolution ``NX=NY=NZ=512`` |br| |CPU_0512_C2CZ_Scal| |CPU_0512_C2CZ_SpeedUp|
53+
54+
* Resolution ``NX=NY=NZ=1024`` |br| |CPU_1024_C2CZ_Scal| |CPU_1024_C2CZ_SpeedUp|
55+
56+
Discussion on Archer2 results
57+
_____________________________
58+
59+
The results above show that the the version 2.0 of 2DECOMP&FFT library keeps on having extremely good
60+
scalability performances.
61+
The transpose tests show no difference between compilers since the tests mainly focus on MPI communication
62+
and for all executable CRAY MPICH (Version 8.1.23) has been used.
63+
It is interesting to notice that a 1D decomposition, when possible, can give up to a 80% speedup in comparison
64+
with the optimal 2D decomposition. This is because of the new feature of the library where a simple copy,
65+
avoiding completely MPI communication, is performed when data are all co-located in the local memory.
66+
This was not the case with the previous version of the library.
67+
CRAY and GNU compilers performances using the *generic* FFT tends to differ for a low core count with the GNU
68+
performing a bit better in some cases (up to 50% performace increase), however results tends to converge with the
69+
increase of the numbers of nodes.
70+
This gives some superlinear behaviour when looking at the speedup.
71+
72+
The **FFTW** has been tested only with the CRAY compiler and it gives a speed up of about 3 for a low core count
73+
decreasing to something in between 1.5 and 2 for the larger number of nodes.
74+
The speed up with the FFTW is generally very close to the ideal lineat behaviour.
75+
76+
..
77+
_Figures for Archer 2
78+
79+
.. |CPU_0512_TrReal_Scal| image:: benchmarks_figs/2023_08_01_Res0512x0512x0512_TrReal_CPU_Archer2_ScalabilityTsec.pdf
80+
:width: 35%
81+
:alt: Archer2 Scalability transpose real test: Resolution 512^3
82+
.. |CPU_0512_TrReal_SpeedUp| image:: benchmarks_figs/2023_08_01_Res0512x0512x0512_TrReal_CPU_Archer2_SpeedUp.pdf
83+
:width: 35%
84+
:alt: Archer2 SpeedUp transpose real test: Resolution 512^3
85+
.. |CPU_1024_TrReal_Scal| image:: benchmarks_figs/2023_08_01_Res1024x1024x1024_TrReal_CPU_Archer2_ScalabilityTsec.pdf
86+
:width: 35%
87+
:alt: Archer2 Scalability transpose real test: Resolution 1024^3
88+
.. |CPU_1024_TrReal_SpeedUp| image:: benchmarks_figs/2023_08_01_Res1024x1024x1024_TrReal_CPU_Archer2_SpeedUp.pdf
89+
:width: 35%
90+
:alt: Archer2 SpeedUp transpose real test: Resolution 1024^3
91+
92+
93+
.. |CPU_0512_TrClx_Scal| image:: benchmarks_figs/2023_08_01_Res0512x0512x0512_TrClx_CPU_Archer2_ScalabilityTsec.pdf
94+
:width: 35%
95+
:alt: Archer2 Scalability transpose complex test: Resolution 512^3
96+
.. |CPU_0512_TrClx_SpeedUp| image:: benchmarks_figs/2023_08_01_Res0512x0512x0512_TrClx_CPU_Archer2_SpeedUp.pdf
97+
:width: 35%
98+
:alt: Archer2 SpeedUp transpose complex test: Resolution 512^3
99+
.. |CPU_1024_TrClx_Scal| image:: benchmarks_figs/2023_08_01_Res1024x1024x1024_TrClx_CPU_Archer2_ScalabilityTsec.pdf
100+
:width: 35%
101+
:alt: Archer2 Scalability transpose complex test: Resolution 1024^3
102+
.. |CPU_1024_TrClx_SpeedUp| image:: benchmarks_figs/2023_08_01_Res1024x1024x1024_TrClx_CPU_Archer2_SpeedUp.pdf
103+
:width: 35%
104+
:alt: Archer2 SpeedUp transpose complex test: Resolution 1024^3
105+
106+
107+
.. |CPU_0512_R2CX_Scal| image:: benchmarks_figs/2023_08_01_Res0512x0512x0512_R2CX_CPU_Archer2_ScalabilityTsec.pdf
108+
:width: 35%
109+
:alt: Archer2 Scalability R2CX test: Resolution 0512^3
110+
.. |CPU_0512_R2CX_SpeedUp| image:: benchmarks_figs/2023_08_01_Res0512x0512x0512_R2CX_CPU_Archer2_SpeedUp.pdf
111+
:width: 35%
112+
:alt: Archer2 SpeedUp R2CX test: Resolution 0512^3
113+
.. |CPU_1024_R2CX_Scal| image:: benchmarks_figs/2023_08_01_Res1024x1024x1024_R2CX_CPU_Archer2_ScalabilityTsec.pdf
114+
:width: 35%
115+
:alt: Archer2 Scalability R2CX test: Resolution 1024^3
116+
.. |CPU_1024_R2CX_SpeedUp| image:: benchmarks_figs/2023_08_01_Res1024x1024x1024_R2CX_CPU_Archer2_SpeedUp.pdf
117+
:width: 35%
118+
:alt: Archer2 SpeedUp R2CX test: Resolution 1024^3
119+
120+
121+
.. |CPU_0512_C2CX_Scal| image:: benchmarks_figs/2023_08_01_Res0512x0512x0512_C2CX_CPU_Archer2_ScalabilityTsec.pdf
122+
:width: 35%
123+
:alt: Archer2 Scalability R2CX test: Resolution 0512^3
124+
.. |CPU_0512_C2CX_SpeedUp| image:: benchmarks_figs/2023_08_01_Res0512x0512x0512_C2CX_CPU_Archer2_SpeedUp.pdf
125+
:width: 35%
126+
:alt: Archer2 SpeedUp R2CX test: Resolution 0512^3
127+
.. |CPU_1024_C2CX_Scal| image:: benchmarks_figs/2023_08_01_Res1024x1024x1024_C2CX_CPU_Archer2_ScalabilityTsec.pdf
128+
:width: 35%
129+
:alt: Archer2 Scalability R2CX test: Resolution 1024^3
130+
.. |CPU_1024_C2CX_SpeedUp| image:: benchmarks_figs/2023_08_01_Res1024x1024x1024_C2CX_CPU_Archer2_SpeedUp.pdf
131+
:width: 35%
132+
:alt: Archer2 SpeedUp R2CX test: Resolution 1024^3
133+
134+
135+
.. |CPU_0512_R2CZ_Scal| image:: benchmarks_figs/2023_08_01_Res0512x0512x0512_R2CZ_CPU_Archer2_ScalabilityTsec.pdf
136+
:width: 35%
137+
:alt: Archer2 Scalability R2CZ test: Resolution 0512^3
138+
.. |CPU_0512_R2CZ_SpeedUp| image:: benchmarks_figs/2023_08_01_Res0512x0512x0512_R2CZ_CPU_Archer2_SpeedUp.pdf
139+
:width: 35%
140+
:alt: Archer2 SpeedUp R2CZ test: Resolution 0512^3
141+
.. |CPU_1024_R2CZ_Scal| image:: benchmarks_figs/2023_08_01_Res1024x1024x1024_R2CZ_CPU_Archer2_ScalabilityTsec.pdf
142+
:width: 35%
143+
:alt: Archer2 Scalability R2CZ test: Resolution 1024^3
144+
.. |CPU_1024_R2CZ_SpeedUp| image:: benchmarks_figs/2023_08_01_Res1024x1024x1024_R2CZ_CPU_Archer2_SpeedUp.pdf
145+
:width: 35%
146+
:alt: Archer2 SpeedUp R2CZ test: Resolution 1024^3
147+
148+
149+
.. |CPU_0512_C2CZ_Scal| image:: benchmarks_figs/2023_08_01_Res0512x0512x0512_C2CZ_CPU_Archer2_ScalabilityTsec.pdf
150+
:width: 35%
151+
:alt: Archer2 Scalability R2CZ test: Resolution 0512^3
152+
.. |CPU_0512_C2CZ_SpeedUp| image:: benchmarks_figs/2023_08_01_Res0512x0512x0512_C2CZ_CPU_Archer2_SpeedUp.pdf
153+
:width: 35%
154+
:alt: Archer2 SpeedUp R2CZ test: Resolution 0512^3
155+
.. |CPU_1024_C2CZ_Scal| image:: benchmarks_figs/2023_08_01_Res1024x1024x1024_C2CZ_CPU_Archer2_ScalabilityTsec.pdf
156+
:width: 35%
157+
:alt: Archer2 Scalability R2CZ test: Resolution 1024^3
158+
.. |CPU_1024_C2CZ_SpeedUp| image:: benchmarks_figs/2023_08_01_Res1024x1024x1024_C2CZ_CPU_Archer2_SpeedUp.pdf
159+
:width: 35%
160+
:alt: Archer2 SpeedUp R2CZ test: Resolution 1024^3
161+
162+
163+
164+

0 commit comments

Comments
 (0)