National Energy Research Scientific Computing Center (NERSC) New User Training, June 2020, 16 Jun 2020

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: 03 Connecting to NERSC via SSH and NX

Description

Part of the NERSC New User Training on June 16, 2020.

Please see https://www.nersc.gov/users/training/events/new-user-training-june-16-2020/ for the training day agenda and presentation slides.

A

See next we're going to talk about a few ways that you can connect to this.

A

So we're going to talk about three ways you can connect to nurse services and then we're going to talk about three more ways that you can connect to corey itself.

A

So the three services that we'll talk about are iris, for managing your account and project help for getting support and my.nurse.gov, which is according to everything, including the current center status.

A

So starting out with iris and clinton spoke a bit about this just before this is the first place that you will go when you get a new account and a very important url to remember quite easy iris.nurse.gov.

A

It looks like this, and this highlighted red bit is very important for if you have any trouble with your account, if you're stuck on anything, this will solve a lot of problems from here. You can reset your password.

A

uh You can get a reset link if you've forgotten your username, using your institutional email or if you've lost your mfa tokens. For instance, your uh something's happened.

B

A

You can recover from here as well, so when you click the login button, you get a screen. That looks a little bit like this. This is our single sign on screen. You'll see this screen with quite a lot of nurse services, not quite all yet, but the majority of services are behind a single sign-on window.

A

The first time that you log in when you haven't when you've just got a new account. This will be all you need, username and password, but very soon after that, you'll find that you're using mfa for everything.

A

So mfa is multi-factor, authentication and you're going to use it a lot at nurse, and I suspect that if you're not already using it a lot outside of nurse, you probably will in the not too distant future.

A

So multifactor authentication means that you're using both your password, so something you know and your phone or some other device or something you have. It means that if somebody finds out your password, that's not enough to hack your account, and this protects nurse users from things like you know this recent uh event, which is something like 11 supercomputers across the world. What happened that nurse was not one of them and we suspect that a you know a significant aspect of why this was protected. Was that mfa made it just that bit too difficult.

A

So, to set up mfa in iris, first of all, what you'll need is a device to get your token from and we're using google authenticator. There is also an option b for if you don't have a smartphone or if you want a backup in case you lose your smartphone, which is authy.com.

A

So it's available. Google authenticator is available in the google play store and also the itunes apple store.

A

There's some links here in the slides, there's also some links in our docs there's quite a detailed walkthrough of how to set it up on our docs pages. So if you go to docs.nurse.gov and do a search in the search box for mfi, you'll find a screen that looks something like.

A

This the next thing that you'll need to do is to set up your mfa tokens in iris, so for that you'll navigate to the mfa. Tab click. On the add token button and you'll get a screen. That looks a bit like this, and with this you can use google authenticator in your phone to scan this qr code to add it there or, if you're using author, you can paste in this authy code down.

A

Here after you've set it up, then the next time that you log in at the single sign-on page, as well as your username and password. So we saw this screen before you'll also be asked for a one-time password, which is uh 60 chips that come from the app with a short video of what it looks like here. This is using authy on the desktop just to be able to see. What's going on so click through the username and password one-time password, select the number from authy paste it in do the number matches.

A

And you're in and once you've logged into one of our single sign-on services surfaces using this screen that will actually cover you for all of them.

A

A few key troubleshooting things, if you find you can't log into iris the most common thing here is that you have a new account and it can take it. It's not.

B

A

Approved it might be that you're just still waiting in the queue or for your account to be available. You can take some time um if you've forgotten your password. You haven't logged in for a long time or you've lost your phone or your mfa tokens in some way. Don't forget, there's links on the irs login page.

A

You can click through on those to solve most of the the most common problems that people tend to have another one that pops up a bit is, I can log into iris, but I can't log in to corey what's going on, and this is because, when you get a new account, you have a nurse account but to log into corey. You need to actually be part of a project.

A

So for that you can check if you're part of a project by looking at the roles tab within irs, and if the answer is no, then what you'll need to do is contact your pi and ask to be added to the project.

A

This doesn't happen automatically at the end of the year that pi needs to remember to re-enroll their members, so that was iris. Next thing is getting help, so uh http help.nurse.gov will help you do contact nurse support.

A

You go through the same single sign-in as you do for iris, and then, when you arrive, you get a screen. That looks something like this. Actually, it's a slightly different color scheme. Now we've just recently had an update that matches the nest color scheme a little bit more closely, but this is kind of what the page will look like.

A

A couple of important links that will help is over here, request forms. So if you need something sort of specific something about your quotas or reservation kind of special requests and you can go to request, forms the one that you'll more commonly use is submit a ticket. So if you're stuck in need help from the nurse consultants, you can submit a ticket and somebody will be with you soon to answer your questions.

A

We're also trialing you can see. Help.Nurse.Gov is our address, it's a little bit small here, but that actually maps across to a servicenow location which is nurse.services.servicenowservices.com.

A

If you add sp after that, just at the moment we're trialing a new service portal. Okay, I don't have a screenshot of, but it's a rather nicer interface that we're hoping will become our default. One.

A

And so for the final means of connecting to nurse services and perhaps the most important my.nurse.gov, which gives you the center status and a portal to everything so you've seen a lot of urls this session already and today and you'll see a lot more. If you only remember one of them, remember this one, because this one will get you pretty much everywhere.

A

When you go there, you see a screen that looks something like this, so this one's after logging in because it's showing my personal disk usage over here.

B

A

Just as a quick snapshot of what you get on there on the dashboard here, you can see how much of your disk quota you're using in the home and scratch file systems over here. On the right hand, side, you can see an overview of the system status.

A

So if you can't log into corey, maybe maybe corey is having a maintenance today and it's not available check your calendar and you can check on my.nurse.gov and see if there's any issues and with a number of the other systems and services as well.

A

Then, over here on the left. We've got a bunch of menu tabs that take you to other important places at nasa, so help.nurse.gov, which we just talked about: service tickets, jupiter.net.gov, which is a very nice product of its own right, and also a way that you can connect to nurse and get a terminal which we'll talk about soon have a bigger session on later www.nurse.gov, which is nurse homepage docs.nurse.gov, which is our documentation highly recommend remembering that one as well.

A

But of course you can get there with my.nurse.gov and irs.nurse.gov, which you've seen a few times now.

A

So most of the information that you see in nurse.gov by my.net.gov does need to log in because a lot of it's attached to your account, so this one's it's a different, um a different setup to the single sign-on.

A

It looks basically the same so you still put in your username your password and the mfa check and use it in the same.

A

Way once you're in uh yeah I'd encourage you to explore this menu of your reports and links on the left hand side here, because there's a lot of interesting information about corey and your useful tips and job history, and so on that you can browse through a couple of particularly useful ones. If you're wondering why your job's, taking a long time to start, you can take a look to see the queue backlog which shows the amount of work. That's queued up at the moment um for different parts of the machine.

A

You can also see what's the typical queue wait times, given a size and scale of a job, another really useful one along here is this job script generator. So, if you're trying to work out, you know what you need to put in your job script, to make it run that can give you a pretty good starting template.

A

So recapping we've gone through three ways to access nuts services iris.nurse.gov for your accountant project, help.nurse.gov to contact, support and my.nurse.gov, which is the address to remember, to get to pretty much everything else.

A

Having talked about nurse services now we'll talk about some ways to connect to corey and again we have three ways that you can do this. You can.

B

A

From a terminal for power users, you can get a effectively a terminal from the browser through jupiter and you can use a product called no machine, also known as nx, uh for making.

B

A

Work a little bit more smoothly, so if you're running assay like matlab or debugger, it's gonna be very helpful. So first.

B

A

Ssh, shell, minus name quarry.ness.gov.

A

If you are this, is this: is the old school traditional method of connecting to a linux server, if you're, comfortable working in a terminal, ssh from your local terminal to quarry, is far and away the most flexible and powerful working environment? For those of us who spent a lot of time in the terminal, find it very difficult going back to something else, most.

B

Importantly, you'll.

A

Need a terminal program: if you use a mac on your desktop or laptop there's a built-in terminal, you can also download a app called item 2, which is an improved version of the terminal. I highly recommend it. That's very nice!

A

If you use windows on your laptop or desktop you'll need putty or some equivalent, so putty is sort of the most common, and I guess traditional way of getting these terminals. uh Xwin32 is also good. uh Git bash is newer and quite a convenient way of getting your terminal. uh This link down here it takes you to putty.

A

If your desktop or laptop is linux, you probably already have your own favorite terminal, and you know how to use this already, if you're, using a chromebook which is uh getting more popular if you're in developer mode, there's a thing called crush, which gives you a terminal access, there's also a little bit more new crostini google for this one, it's linux in a container that runs on your chromebook.

A

Yeah it gives you really quickly and easily a linux environment and your chromebook, it's great or you can download, or it might be already included the ssh app and use that to connect.

A

So when you connect, this is what it looks like this particular screenshot is from itunes 2 on a mac, so you'll type in ssh dash, l for login name, and then your nurse username, uh we'll come back to jashy.

A

Corey.Nurse.Gov is the name of the machine that you're trying to log into and the first time you log in you'll get a warning. Something like this. The authenticity of host corey.nurse.gov can't be established key fingerprint. Are you sure?

A

So? What does this mean? Ssh prints this message when it doesn't recognize the computer that calls itself corey. So if this is the first time you've ever logged into corey, that's to be expected, it's just meeting it. It doesn't recognize it yet if you've been logging into corey every day for the last week, and then today you get this message: that's a red flag, something something's gone wrong. Why? Why doesn't cory? Look like it used to so when you see this message, uh what can you do? How do you?

A

How do you decide whether or not you can continue connecting this is where this uh fingerprint is quite handy, so each linux system has kind of a unique fingerprint. It can be only generated by that system and the way that your laptop through ssh decides that it can recognize cory or not. Is it compares the fingerprint against the one that it kept from previously every now, and then these fingerprints do change, particularly just after a maintenance.

A

In some cases, the update to quarry will result in a different fingerprint.

A

So to give you a secure way of making sure that you're connecting to what you think you're connecting to we publish the key fingerprints for nurse systems on docs.nurse.gov, so it says this web address which will take you there or you can just go to docs.net.gov and in the search box type fingerprints and you should find it so once you've gone there, you'll see a page that looks something like this: the corey fingerprint and you can kind of eyeball it.

A

It looks like the same fingerprint yep, that's good. I can safely log in if it's a different fingerprint, please contact us. That means that that most likely means potentially means that somebody is out there trying to and we to stop that.

A

So when you're, connecting with ssh, your ssh in uh lots of text will go by with you know, disclaimers and and what have you and then you'll see a screen that looks a bit like this. It says log in connection to host quarry something password plus otp.

A

So what you'll do here is you'll enter your iris password, followed by the digits, the six digits that came from google authenticator with no spaces in between. So if your password is qwerty, which I hope it isn't and google authenticator has these numbers you'll type, all in a single string 2687921 and as you type nothing will appear on the screen. It'll look like nothing's happening. This is a little bit disconcerting, the first time you see it, but it's actually normal. This is the way or a way that ssh.

A

Adds extra layers of protection to your password, it doesn't even show how many characters are being printed.

A

If you get a a login prompt that says password that doesn't say anything about otp, what this probably means is that your account isn't ready. Yet so it.

B

Hasn't asked for your.

A

Otp because it doesn't know about it, which in most cases means that it's not there.

A

Right so we kind of glossed over that ssh dash y option. What's what's that all about ssh takes a couple of options: either dash y or dash x. Dash y can be a little bit more reliable, and what this does is that allows x programs x is the name of the linux gui system to run and display on your local screen.

A

So because your core is somewhere yeah deep at nurse. Can your local screen is on your desktop, so there needs to be some sort of a system where, if you're running matlab on corey it can draw in your screen and x is the protocol that allows that to happen for it to work, you need an x server, so the x client is the program. You know matlab or debugger, that's running on corey and the x server is the window, that's drawing the x windows, and so that runs on your laptop.

A

If you're, using a mac x, quartz is good if you're using windows sigwin is good. Here's the catch, though, if you're outside of the building and nest and you're on a slow network, it can be quite painfully slow. So we're going to talk about some alternatives to that failure. Scene.

A

After you've logged in a few times, you'll notice that you start getting a little bit tired of typing your password and looking up the one-time password every time. So there is a way to make this a little bit easier. So we have a tool called ssh proxy dot, sh, which you can find by searching for mfa, ssh docs.nurse.gov.

A

What ssh proxy does is. It makes a short term certificate that lasts for 24 hours, so you type in your password and one time password just once, and then you don't need to type it in again from that machine for the next 24 hours. So this helps to give a balance of both the protection of mfa, but um significantly more convenience.

A

So this is what it looks like after you've downloaded it and set it up. If you're running from a mac, for instance, you'll type ssh proxy.u and your username, it will ask for your password and otp. You do exactly the same as different ways at the log input of quarry one. After the other, no spaces, nothing will appear at the prompt and it will set up a key and it will put it by default. It will call the key nurse in this hidden.ssh directory.

A

You can then use that key to log in so notice now ssh dash, I and the path to the key in corey notice, you're, not putting your username anymore you're, just using the key or simpler. Still, you can add it to your keychain with the dash a option so ssh proxy dash a and your username login, and then in the case of a mac at least it will remember who you are and what your ssh um certificate is and then all you need to do is oops ssh dash, l, your username corey.net.gov and the wall.

A

This is yet with the key that's saved in your keychain.

A

If you're using windows, we uh now support windows with ssh proxy, uh the executables called sshproxy.edu for detailed instructions. Again, look at docs.nurse.gov do a search for windows ssh proxy.

A

To use it you'll first need to start a command prompt which you can do by typing cmd in the search box on windows, and this is the sequence of commands. You'll use, so you'll run ssh proxy with your username and then so. This is using putty to handle the key management.

A

There's a tool within putty called p agent. So when you run ssh proxy, it will give you a key which will be called something like nurse key dot. Pbk use p agent to pick up that key, and then you can run putting using that agent to log in.

A

So let's say you come in through a terminal kind of the traditional way. A newer way. That's very convenient is to use jupyter and jupiter gives you both notebooks and also terminals through your browser. So you can use your browser as your sort of point of contact for everything. We have a more detailed session on jupiter coming up soon, but I'll do a quick walk through first of how you can access the terminal through jupiter.

A

B

Can access jupiter.

A

And corey, via by putting your browser at jupiter.nurse.gov, you'll, see something that looks like this. I notice this is a little bit similar to the minus one. It's not the single sign-on that iris uses, but it's the same principle. You put your username your password and the otp from google authenticator that will take you into jupiter hub and you'll have various options of what you can start and in most cases what you want to do is you want to start a session on corey on a shared cpu node, because this should be your default.

A

So this starts a session on uh something that's um pretty much the same as the.

B

A

Nodes, it's a fairly beefy server that has a you know, direct connection recording once you've logged in you'll get this launcher with all these different things that you can start and the one we're going to talk about here. Is this one right down the bottom under other? You can start a terminal, and it will look like this and it's exactly.

B

A

Terminal that usshed into, but you haven't, had to jump through the ssh tubes, so particularly if you're just starting out or you don't have a terminal set up that you're comfortable with and you're on the occasion you need to log into corey to use the terminal. Jupiter can be a great option for that.

A

Finally, no machine for if you're running, gui, apps and you'll notice that the web address here says: nx uh no machine and nx tend to be used interchangeably. I think the history is that it used to be called nx and it changed its name to no machine some years back.

A

So if you're running a gui app such as matlab or a debugger such as ddt or v2 and performance analysis, if there's a quite handy, they give you a nice graphical interface, but over a long network they can be painfully slow. So we're going to talk briefly about why this is and how we can fix it. This is referencing back to that. This is h, dash y that we talked about before.

A

So the reason is that, when you're running a gui application on this, you know distant um quarry system you're using a client server kind of architecture where the gui application, which is the x client, is running on one system and it's sending messages to the x server, which is the window manager which is running on your laptop and the protocol that it uses the x protocol is very chatty. It sends a lot of traffic and you know if you're on fast network like the one internally at nurse.

A

This is fine, you don't notice it, but the internet out there is not all a fast network, and so all this traffic goes outside of nurse hits the internet and just jams up like the bay bridge in the morning and it's uh you know it can be really quite slow. You know the further the further you are on the network, the more painful you'll find the gui experience.

A

B

A

No machine solves this, is we actually run the windowing system on a virtual machine inside of nurse? So it's got a fast network that the x protocol can talk over and then nx uses a much a slimmer, faster protocol to communicate with the client. That's running on your laptop.

A

This has also got another nice little side effect, which is the internet out. There is a little bit fragile connections break. Sometimes the wi-fi might fall over in the normal course of things. If you're running your x server and your laptop and your network falls over that bricks, your application so yeah map that will quick with nx, because you're running the window manager inside of nursk. The part that's at risk of breaking. Is this part out here and okay?

A

Your connection to your session falls over, but it's still running inside of nurse in this virtual machine you can just reconnect it and the window comes back up and you've got your application still running exactly as you left. It.

A

So, there's a few steps involved in setting it up. We have detailed instructions at this address on docs.nurse.gov.

A

You can go to the search box and look for nx or node machine, and you should find it fairly really easily.

A

In a nutshell, you need to download the client make sure that you get the client, not the server or the workstation you're, trying to run the client on your laptop.

A

Then you'll need to set up a connection, and we've got a video tutorial that you can watch for that and then once it's set up, so you can set it up to use this ssh proxy, but you don't have to in the example with if you're, not using ssh proxy.

A

The way you log on looks like this, so you'll get a log on screen which will ask for your username and password, and this is where this is the kind of the trick to remember it's asking for your password, but it actually needs both your password and the mfa. Token.

A

uh Is it because the nx client, no machine client, doesn't actually know about mfa, it's kind of hidden behind. So you put your password and your numbers here. No no actual text will appear. It will just be dots. This is just an example of what it looks like very important.

A

Don't save this password to the connection file, because this password includes the one-time password from google authenticator, which changes every 60 seconds. So you know you don't want to save that to your connection file and try to reuse it, but it won't work.

A

Once you've logged in you'll get a screen that looks like this, so you can create a new virtual desktop. If you already have one running, it will give you the option of connecting to it and when you connect to it, you'll see something kind of like this so you'll.

A

This green button up here opens a terminal window on corey you'll go straight there and just to show off that you can use this for x. Applications x term is a x terminal.

A

Type x term started in the background, and you see this with a pop up and that will give you a much smoother faster, nicer environment for running x, programs.

A

So we've run slightly over, but uh I think we were scheduled for a for a break to what we've covered in this session. We talked about three ways you can can connect to nurse services irs, which is your first stop for managing accounting project help.nurse.gov for your contacting support and my.nurse.gov for one url to remember for center status and a portal to pretty much everything else.

A

That's the nurse services. We've also talked about how to connect to quarry itself. You can use ssh through a terminal. You can get a terminal in your web browser via jupyter.net.gov or if you're, using gui apps. Setting up no machine client on your laptop and connecting through no machine is highly worthwhile.

A

So that's all for connecting to nest for today.

B

You got a few questions. Most of them are answered by the staff already on the google doc on two more here just for you to answer so when using ssh proxy.sh, what does the dash a flag? Do.

A

So the dash a flag adds it to your keychain, so if you're using a mac and you're familiar with the keychain, what it does underneath is it starts or connects to the ssh agent, that's running on your laptop and adds the key to that agent. So it basically tells your laptop to remember this key for this session, so you don't need to tell it to use this identity.

A

So so what that saves you from doing is typing that uh dash, I to which I'm back to that slide. That looks like.

A

So if you don't add it to the keychain, then you need to explicitly tell ssh which identity you're logging in as by pointing it as the ssh key that says, h proxy generated and then it to your keychain kind of does.

B

A

Automatically yeah, that's the identity too, for glory to what your laptop is already remembering.

B

Okay, let's save the other question um to be answered um on google on google doc by written answers so that we could have more time so it's related to jupiter and nx, or ask the jupiter experts to answer that. That's also a jupiter session in the afternoon.

B

So let's take a.

B