National Energy Research Scientific Computing Center (NERSC) New User Training 2019, 14 Aug 2019

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: 4. The Cray Programming Environment and Compiling

Description

Learn how to compile your codes for Cori.

Slides for all sessions can be downloaded from here: https://www.nersc.gov/users/training/events/new-user-training-june-21-2019/

A

So this talk is about the basics of compilations, so I will not talk about any building systems like like make, and you know she make and whatever the pack building systems you use, but I will only talk about the compile and incline and how you take care of those on curry.

A

So this is the curry system configuration. It is a cray at the c40 and in general. What do you you can just think of K&L has a lot of more weaker cause in comparison to Haswell processors, but they are also. They are both Intel processors. So, in the context of compilation, one thing you need to remember is the K&L supports more instruction extended instruction sets. So if you build binary that optimize the for Sol architecture, it may run it runs on K&L, but not vice versa.

A

So you need to remember this so in general, for the optimal performance, we recommend you to build separate binaries for each type of processors, so I will go through. I will use the several slides to go through the general thing. You need to know about compilation on curry, so we have three supported program environments, so they are Intel, gano and Cray. So here among these three, we have inhale as a default. So by programming environment we mainly means the compilers and the libraries that that are built with matching compilers.

A

So Cray actually did a lot of work to pack these things together and to provide a user friendly programming environment and they provide so called the compiler wrappers. Something called FTN see: C &, C capital, C C for Fortran and C and C++ compilers respectively.

A

So the the compiler wrappers interact awaited emoji of the loaded in your environment and take care many things for compilation, so your task of compilation is very easy, so one thing I want to mention is: if you use compiler wrappers, those are include pairs and also libraries can be taken care of automatically and we needed to know. If we use come compiler wrappers to compile, we are doing cross compilation. This means you log into curry, login, those those are has well knows, and then you compile for computer notes.

A

So, as we talked earlier, we have a two types of computer nodes so as well and the K&L. So by default, if your login to curry login nodes, you can see crepe here as well, module is loaded. So if you don't, you do anything, use the compiler wrappers to build a binary that is optimized for Haswell okay. So this is the environment. You see when you log into curry, so you can see crepe here as well is loaded and then also you can see this.

A

Prg e MV intel this is a programming environment, module, so false tree supported modules. We have corresponding modules available, so you can also see the Intel module is loaded, so your default is Intel program environment so to compile for career as well. That's easy! So we let to say this is just a demo, so you have just a single source code there and then you use FTN. This is a Fortran compiler wrapper, so you use that compile and you got a that out. You just run it on the computer.

A

No later we will cover these in next talk. But compilation is that easy? You just you don't have to worry about to provide the libraries and headers and those things, but you will see like what entered in this compilation line, but anyway the the simple I mean the easiest thing: it's just to use the wrappers and compile your code, and if you want to use control compiler you test it before you compile. You just run this module, swap this program environment module.

A

Then you will use kernel, compiler and all the libraries that that were built with the new compiler. So that's what we do on help as well, so one thing I want to mention some users and we had some users confused, that they thought these compiler wrappers. This FTA NCC and CC are correct compilers, but they are not so there's just the wrappers to I mean they wrap those native of compilers. They can wrap Intel guru and Korea compilers. They self they are just the wrappers, so they are not equal to Korea compilers.

A

So to compile for curry, K&L and all you need to do is before compile compilation you just the swap of this creep here as well: creepy Mike K&L. So if you do this, then the K&L architecture, specific compiler flags, will enter your compilation line automatically. So you build a binary like this, then that one will be optimal for ok or now.

A

So, if I have only just these few minutes, then yeah. It is all about how to compile on curry, but we can extend a little bit just to talk about more what you I mean it can helps you help you to you, know, understand better. So now, I come to data compiled ink line. So from there the first one is: you need to choose compiler so regarding which compiler is the best for you. You need to to experiment and do benchmark by yourself using your application. You need to run.

A

You need to take choose by yourself, so in general Intel, if you choose to use Intel compiler, then you have a better chance to get the processor specific optimization. So this is Intel processors they you can imagine like they must have done better job to optimize.

A

You know code for their own, you know processors, so this is understandable, especially for the newer processor type or something like K&L. Here you can think that or Intel might be, might have done better job and then, if you want to use correct, compiler many new features, this is very up-to-date features. I mean it's it's good to have many up to today to new features and also a lot of optimizations so especially creates a strong over there of Fortran compilers. So you can try this Korea compiler.

A

If you are care to explore the compilers and then go new compiler is free software. So you can see many the open software they package the way the kernel compiler. So our recommendation is just to start with the compilers, whatever used to by the vendors or the the coded developers. That way you will minimize the chance to run into a compiler or code bugs because developers may you know, may not test every compiler.

A

So it's a better, it's better for you to get started over there I mean the compilers they used regarding the compiler flags, so you can check each compilers main page, something like you can do. Man I thought this is a for Intel, Fortran compiler. You can find so many compiler flags, including including those optimization flags, but my recommendation is just to go with the default. So let's say you are new to our system and you want to compile some code.

A

You should be more conservative because you don't know you know usually like if you go with a higher optimizers and there is more chance. It's embargo to get involved. I mean introduced, so go with the default, which I believe is more extensively tested and then something like Intel its default. Optimization level is OH, OH level of optimization and Korea is also they claim or but they are a little bit higher than OH somewhere in between Oh and oh sorry, level and then go no.

A

You need to provide OH are manually because by default they don't put any optimization flag. So you need to put this and then for the open, MP code that you need to provide this flag for intel, something like a queue openmp. You need to put there and I want to recommend this for both options. So it's a compiler. They supported the verbose option and they provide a lot of information under the hood. What the compiler does for you.

A

So it is a very good thing to put it there to learn what the compiler does under the hood, so usually regarding using what the compiler Flags is good, especially the optimization flags. I just recommend the core more conservative side and then do the validity check first, once you are comfortable, these things are generate character result. Then you can explore long higher compiler optimizations. So one thing I want to remind you: is the default behavior of compilers? Could be different, especially one example.

A

Here is the default number of OpenMP threads used by the different binaries, but by the you know, are different. The compilers are different, so something like a canoe and some compilers like renewing and Intel they just let's say you build one binary called a dot out and then run it just to say a dot out. You didn't, especially if you don't specify the OpenMP number of Suez by default, it would use all the CPU sled CPU slots it can drift into. So in general, we have to say on all.

A

We know that you can try this to you. You build something and run it. It will grab all the cores available on the on the login knows so the default is different. There, like a cray compiler, it just default to use only one OpenMP threads okay. So that's a compiler flags. So next, let's do the let's look at how to take care of this header and the library past, and also the libraries so in general, I suppose you're all familiar with how you do manually.

A

So it manually you invoke native a compiler call and then use I capital I and then put to the test to the headers and then use a capital L, and then you put pass to the libraries and then use a I'll provide the library file name there. So this is general practice. You need to do this manually in, if know like a cray, even if there is no nicely packaged the environment is supported there.

A

So that's the manual way, but on create system on quarry we can do it automatically and then provided you have to use their create reppers, the FT, NCC and capital CG. That way, this header and the library past, and also libraries can be taken care of automatically. So you don't have to bother with provide those by yourself.

A

So I will use the next few slides talk about compiler wrappers. So so, as I said, the compiler wrappers they wrapped native a compiler call and then do many things for you so, depending on the invite in the environment, what kind of homology was loaded? So it will pick up. The I mean the libraries and those passes automatically from your environment and then put them in your compile inclined. So one thing we needed to know is I mentioned this, but it does cross compilation. So I want to mention a couple of things.

A

So if from the login node, which is Haswell node and then you compile for another computer, know that the Haswell computer node, it's a nothing indeed, it's not matter it just doing it's just like you build on that note, it's fine, but if you beautiful K&L, then some of the packages they they're configure script. Usually they do like a compiler check. They try to see if we are compiler works or not so the way they do this is they just build a small binary and then run it on the host machine that you are.

A

You are you know, type of compilation you are compiling on and then see. If they are, the binary runs, they need to say: okay, compiler works, so many configure scripted as this, but for the cross compilation, as I mentioned earlier, the K&L binary will not run on Haswell, so it will fail. The compiler check, so in that case you can try this a host equal to some, like a general processor type. Here that way you can get around that walk around work around that error, but the some packages doesn't support this option.

A

In that case, you may want to go onto the computer, node and compile. In that case you can just grab a one: node use the cell log and then compile on the K&L node, so compiler wrappers link statically by default.

A

This is a crazy preference because they are better for coda performance as but we may change, I mean we will be quarry system will be upgraded, attune another OS in end of July and by the time the default will be changed, Willis a total dynamic, but still now we are build binaries by default they will be linked statically.

A

So if you want build the dynamic by net, if you want to link your application dynamically, then you need to provide this option dynamic option in your compile, inclined or you can use this environment variable set to this, and then you don't have to change the compile inclined.

A

One thing I want to mention is: if you run your code at scale like use a lot of processors, then the dynamically linked the library binaries may take time to load up your library share the libraries, so it may be, have some performance heat, so we recommend for those the largest scale run.

A

You should choose a static build, so we can take a look what the compiler records does for you, so it does first it as like this architecture, specific flag for you, so you don't have to manually, provide this to be able to optimize the binary and then, additionally, it has that headers passed and the library passes in the compile, inclined and automatically also putted the libraries ER on them and also the good part is it does a lot of things for you, but if we want to overwrite what it what it does, then you can provide you the option there.

A

Then this one would take precedence, so I I want to just provide verbose output from the compiler wrappers.

A

So I really like this, because it tells you a lot of information and you can learn a lot from these verbose output. So this is the example like I log in to hot curry and then I didn't. Do anything I want to compile a small code that uses the same library call then I just do ft N and minus V and then my source code. It gives me a lot of information and then it tells me what the kind of libraries are linked into. So you can see those capital L.

A

Those are the places where the libraries get searched and then all the L thing is the libraries enter the your link line and then I put to this a small link option here: WL and comma y Diez M underscore this is a user for options. When you have multiple options, let's say our default: by default, we load Korea leaves a library which provides the same function inside so sometimes you may want to overwrite there. You want to use m KL, but you don't know exactly the code.

A

Indeed use the m KL or use the leaves I. Then you can just put this option there and then you can find you can see the down below. You can see it's covered here, but it tells you where the definition come from. So in this case it tells you that these iam come from. This leaves I because I didn't do anything, but if you want to overwrite that uses something like m KL option, then it it will tell you. It comes from m KL, okay, so now then I think I talked about the compiler wrappers.

A

Then we can talk about available libraries. So if you're logging into Korean type emoji- oh well, you see really tons of literally tons of the modules and available and many of them Cray provided the libraries. So how you tell it's a a nursery step also support a lot, so you can tell which ones are provided by K and which ones provided by nursing staff when your type of module- oh well, you see if those module fire come from.

A

This opt crate, this type of a path, then those are provided by Crais and if you see the modules come from so less users, less common. Those are nurses, staff supported, libraries.

A

Okay, so sometime users need to know like where those libraries are installed. Then you can just follow steps here. You do module shell and module name, and then it will tells you the installation path. So in this case take this crate impeach example: you see module, show Korean pete's, you see a lot of output. Basically it tells you what this module defines or what this module does and inside you can find this Korean pitch dir. This is the directory where the module this library is installed, and then you always get to the installation directory.

A

You can find the include path and you can see included and the live directories. So you further LS two data live directory. You can see all the library files and then also areas that include the directory. You see all the headers there, so this is how you figure out where the libraries and header files are so in case. You need to manually provide these paths to your configure script. Then you can try this way and then next few examples will be telling you how to link to the specific library. These are the examples I included.

A

More than enough to here try to show you how to link to those commonly use the libraries. So, for example, if you want to use grey, MPI and career leaves I. Those are already loaded by default. So if we all call the let's say, I put this a parallel hello here, but let's say inside I used like a scale APEC and whatever those palazzo routines and those are provided by career leaves I. Then I just go there and then sorry just compile.

A

Then it will automatically link to MPI library and also Korea leaves I, and if we want to use this hdf and netcdf library, you just say modulo the cray, the hdf5 and then just compile, then the the IO hdf5 libraries will be automatically linked into, and then this is another example like a pet to see and fftw.

A

Basically, it's just all you need is just the load of the module and then compile just the compiler invocation. You use the wrapper and then the source code, and then this is an example on how you use nurse provided the library modules. So some of the nurse commodious actually follow the create convention. Then they can interact a way to the wrappers. So you don't have. You can do the same thing. Something like alpha.

A

This is a nurse provided the library, but you load that and then you don't have to manually, take care of the library thing here, but it will do it automatically. For you, sorry, this thing is very sensitive to the touch. I'm, sorry and then the way you tell if a nurse provided the module can do this automatically or not. Is you just look at module module names, output and just look for this pkg configure lives. There are these three environment variables in red. You look for them.

A

If they are defined, then those modules interactive interact with the qurían wrappers. So you don't have to take care in the command line in your compile link line, but many of the nurse provided the module file. It doesn't follow the don't, follow the Korea convention, so you need to explicitly provide those headers and pests and libraries, but the fortune usually for your convenience. We provide the environment variable something like a DSL. You see here, you load the module and this environment variable is defined which expands as this.

A

It has included health library paths and also the library names so something like mkl. We recommend you go to the linking link line advisor that Intel provides or you can just if you use Intel compiler, you can just use this mkl compile link line option, so you put there that flag. Then it can automatically linked to the mko library. Again, you can use this verbose option see what is the name, what other libraries included in your build if you use this flag so for a new compiler?

A

If you want to use mkl, you need to give this pass where the library is, but the easiest thing is, you can save this environment variable called mkl root and then use it and provide these libraries explicitly.

A

Ok, so, as a summary, I think we can mention three compilers of program. Environments are supported and then use compiler wrappers whenever possible wherever possible. This is the important message so I read it and also made it bold. So this is actually the most important message of my talk so whenever possible, just to use compiler wrappers there to make your life is much easier, okay and then to compile for K&L one more step.

A

You need to swap this creepy has well Mike care now and then you can use the men, I mean many libraries are available and then you can use them and if you use compiler wrappers and you scrape provided libraries, that's very easy just to load, and then you compile and then to link dynamically. For now you need to provide this dynamic option or you need to set this link link crepey link type dynamic option there. You need to set that and then another important message of my talk is learn from compiler, where both output.

A

So this is a where you know you can learn a lot of information about compilers. So I strongly recommend this and then recommend the readings I would like to. In addition to the nurse website, I strongly recommend to get familiar with reading compiler main pieces. You can learn a lot from them and actually many things almost all things you can come from the compiler main page and then also some time. You would like to read this linker main page. Those are also very useful before I end.

A

The talk I have just a short test, so if I say correct, compiler wrappers are correct compilers. Is it correct.

A

Right so I just want to make these things and create compiler wrappers, it's just a wrapper and then under the hole that they can call different compilers. They can call Intel, canoe and Cray's correct, compiler right, so I just want to make sure you have this knowledge and then you can comfortably switch between compilers and compile on curry. Thank you for your attention.