Magento Cloud Community, 6 Aug 2020

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Magento Cloud Demo - Overview of 2FA functionality with Billy Gilbert and Matt Johnson

Description

In this demo Billy and Matt will cover the next topics:

- 2FA on Magento Cloud
- Aggregated logs in NewRelic

A

Awesome so my name is matt johnson uh joining me, of course, billy gilbert, that everyone knows and love. We are going to be talking through uh two new additions to uh the cloud catalog.

A

uh First off we're going to be discussing a magento commerce cloud 2fa for ssh and then after a quick demo on that we'll be going on to uh new relic logs, which has also been released so first off for 2fa for ssh first off. Why are we doing this, and the tldr of everything is that over the past couple years, we've been seeing an increasing number of cloud customer stores being compromised due to ssh mismanagement and ssh based attacks.

A

um It's really become a big attack, vector and one that we wanted to really plug up and get in the green.

A

So to be clear on what 2fa on ssh is it's not necessarily about using 2fa during the ssh login itself, but about ensuring that the credentials used for ssh have been quite acquired by a means of a flow that was protected by 2fa, and what that really means is logging into your cloud. Account via 2fa really covers that mechanism and so generating that access token.

A

That has the parts necessary to ssh into those containers.

A

As I said, that user must have logged in into the cloud ui after using 2fa and so you're going to want to make sure that 2fa is enabled on accounts.magento.cloud in order to be able to use this functionality at all and the access token that's granted, as part of that, then keeps track of that authentication fact, and that token is then used to generate these ssh certificates.

A

Now, from a broader standpoint and from our merchant and si community standpoint, 2fa is project level, and so what that means is that once 2fa for ssh is enabled on a project all users with ssh access to that project or to ssh access that project will need to go via 2fa.

A

So it's not just user. It is for a project. Very much want to call that out out of the gates. 2Fa for ssh is not enabled, is default for all projects. So it's something that project owners are going to need to request.

A

Project owners need to submit zendesk tickets to request enablement of the feature, and while I was going to uh update the second bullet point here, it's it uh or I thought I did so sorry about that. But what this really means is platform. Admin is a role construct for internal adobe, magento users.

A

It really says anyone who has admin access internally. Two projects is going to have the 2fa for ssh enforced on them, so there are no internal adobe or magento employees that are able to ssh without this functionality. At the admin level, we do support machine based account usage via use of an api token. This can be generated in your account's ui on cloud and at this point I will hand it off to billy to go ahead and give an actual walkthrough.

B

Hey good morning, everyone I'm switching to my screen share. So uh please, let me know if you're seeing my screen with the thumbs up but I'll go ahead and dive in here, so yeah we're gonna do a demo now. So um what you'll see? What I have up on my screen is what will happen if you try to ssh into a project that uh has uh ufa enabled, but um you have not authenticated right. So what I'm? What we're seeing here is. This reason access requires mfa.

B

So um so, like matt, said, there's a few key points here. So the first thing you need to do is um authenticate through a uh through the magento cloud 2fa um to the magento.cloud site. So in this case I'm going to just do ms mvc login, which is going to give me the browser login here. So I've already authenticated to magento.cloud, and it's got my um my 2fa cached since you have the option to remember that, but just to show where that's set up.

B

If you go to your name and then account settings, um you will see a section on the bottom here uh for security, where you can set up a tfa application. So um you can also see that I have trusted browsers here, set to you know not expire and then the application itself.

B

So um now that I've logged in um there's a configuration, I have locally, that's already generated the certificate. So what you'll notice now is, if I attempt the ssh command again.

B

It is going to allow me, through to the site.

B

So uh you'll see I'm now able to authenticate now um I want to show a few of the configurations here as seen a little bit of the inside uh scoop as to what's actually happening. So if I go look in my actual ssh folder now, um you'll notice that I have a config file here um now. I already previously had this generated uh for a few commandments, such as putting in my identity identity file. But what you're going to notice here is that it has added these last four lines to my ssh config file.

B

So um in this case it's just going to be including additional configurations but you're going to notice that this is going to be in that magento cloud folder. So let's go ahead and take a look at that.

B

All right so excuse me all right, so we take a look here. What we're getting is the session.config file, so if I go ahead and put uh out the outputs for that, um you'll notice, uh very similarly um we're getting uh this session config right so again, we'll go. Take a look at that file.

B

And that should contain the meat of what is actually being modified here. So um what you're going to notice here? We've got an automatic refresh, so um you know in the events uh you know we need to regenerate it that go ahead and takes care of it for us, so that's going to make it so the circuit will continue to be generated and then what you're going to notice that it's going to actually add this additional um certificate file right and as well as identify identity files.

B

So this uh this key right here this this private key, is generated by the cli um so that you will have that available. Then this cert is the actual ssh certificate generated, and then, of course it does add my additional or my my my default private key that I've generated for my accounts. That'll stay there as well.

B

um The one thing we do recommend is: if you are having any issues with ssh, um go ahead and run the command with the dash dvd option. The reason I think that's really important is you're. Gonna see the identity files that it's trying to generate or trying to use. um So, of course we can see here, it's you know using the certificate um and then, of course it accepted it.

B

So, uh if you're running into any issues, particularly if you're using you know going between, um you know, accounts that include uh 2fa having enabled and not go and check those configurations.

B

um The other thing that is a fairly new rollout that we've exposed is the ability to authenticate by an api token. So I I won't show that for security purposes, but um if we take a look at the off space or the authentication space, you'll notice, this api token login now on your user account. uh If you take a look, you now have an api tokens um and from here you can create an ip api token that you can use to log in um the thing that's very convenient about this.

B

Is that if you generate this token, while you are authenticated with 2fa, it will maintain that level of access. um So if I were to go ahead and do that uh I'll get it'll, ask me for the token I would enter it and then it would be considered a 2fa authentication.

B

This can also be used to create machine accounts. So if you have any type of automation um that that you use that, can um you know that that can be used to automate ssh scripts?

B

um A few other things I want to show is this dev docs link. um So it's got some really great examples. um Of course everything we're discussing here, the one that I want to show in particular, is this uh magento cloud config.

B

So if you create this file and put this configuration in there, it will actually create a certificate for you every time you log in so you'll be able to skip that step. um Of course, it still requires the you know the 2fa login to magento.cloud, but this is very helpful, particularly if you're doing any type of automation around ssh.

B

The other thing we like to point out is that this also impacts uh git, which is of course done on magento cloud through um uh through ssh. So you know be aware: if you're trying to do any, you know get operations, you will need to authenticate with with mfa as well, um and then just the final thing I want to show is the way you can view if a project is configured for mfa, so um this is going to be my my project.

B

So if you run this command, um this curl command with to the slash settings, endpoint you're, going to get a list of the configurations for your project, but what you'll notice here is: there's an enforce mfa. Now, of course, as matt noted for all internal users, we always have to use mfa. So as a as an adobe employee, I have to use mfa for any project, but you'll notice. This particular project is not directly configured for mfa.

B

So um so this way you can double check a project and then, of course, as matt noted, uh if you do want wish to enable this on a project, you will need to request that through support. um So with that, I'm gonna turn it back over to matt for the next portion of our presentation.

A

Well, thanks billy.

A

So I'll keep this short and sweet, so we can get to the the meat of it. But uh the next piece here, as I mentioned, is new relic logs, which was just released in conjunction with 2.4 last week. um You do not need to be on 2.4 to use logs. This has been added to our new relic suite of products. So now, along with apm and infrastructure, you also have access to the logs functionality within logs. We are aggregating our application, our cluster logs, as well as all fastly, cdn and waft logs.

A

So you have a full data set to actually query against.

A

And with new relic we're going to really go through the querying to search aggregated log data so again across the app infra and all fastly logs visualizing this log data and how to work with that interface. And then you do also have the ability to create custom charts and dashboards and alerts.

A

um Alerting is a is a lifesaver, as many folks often know- and this is really part of the narrative- that cloud has really been undergoing over the past year and a half on uh transforming from a black box into putting more control in the user's hand. So having you need to rely less on us for visibility and you'll be able to control that as much as possible.

A

You can uh see our dev docs for some more information and we will be updating that in the future. As we start populating with uh recommended, queries recommended use cases as we move forward on this, and you can also visit a new relic documentation for the uh full list of functionalities as well, and with that I will hand back over to billy to actually go over the logging functionality.

B

All right, so I stole the screen share from matt. Hopefully there's the button all right, so hopefully you all are seeing uh the new new relic one landing page. So um up here, I've got one of our internal accounts. This is actually our pwa demo account um so I'll sort of show the the key screens here that you're looking at for logs. So um of course the first one is the logs ui.

B

So um this this screen is particularly useful for um just being able to see a log stream so um give us one moment to log or to load up since I have a deeper level of access, there's a fair bit that it needs to load but um yeah. So this this screen in particular, is going to be able to show us uh all of our logs, whereas uh there's a separate ui for actually doing more deeper. You know querying and aggregating, and things like that, uh as well as neural insights.

B

So um one moment I'm gonna try to refresh this to see if it will load better for.

B

Us one moment please: well, actually you know what I'll start with the inside screen. So um this is, you know, of course, my relics uh legacy product. um However, I like it because it gives you some some deeper functionality.

B

So um in this case, if I go ahead and use what urela calls nerc will cinderella query language, um I can just select star and see all of my logs here so um in this case we're getting some basic information, we're seeing you know which project that it came from or which cluster, rather um uh you know the source of it um and then eventually over here we're gonna see the actual message. We also have this like denfield uh tracking. You know where it came from so um and then of course, yeah the message.

B

So it looks like here we're getting a mix of access, php access logs, and it looks like a support report so um sort of the key that we used to differentiate between the the logs is, you know again, as matt noted, we have uh the fastly logs and then, of course, the actual cluster logs.

B

So the simplest way to look between the two of those is to um you know, check for a field that only exists in one or the other, so um with the cluster logs using the uh the rsys log specification, um I'd like to just check for this ident field. So if I throw an ident, is not null on here and run that query, I'm only going to get cluster logs um uh neural does not quite the normal query. Language doesn't completely match sql, so it does have some nuances.

B

So in this case, ins uh facet um is going to be basically a group by. So, if I do a facet ident, this will show me all of the log counts uh by file name.

B

So in this case we're seeing you know, I've got a thousand prong logs, et cetera, et cetera. Now, if we want to go and just look at fastly logs um again, I like to use identity sort of a pivot, um so I do is null and then, let's do let's say faster by client ip.

B

So this is the way you could um review traffic by ip and look for um you know people performing at ddos attack or something like that, um so it does look like we have the more modern ui loaded now um like I said this is a very nice clean ui for just looking at streams of logs and again to muralik one. um You know as much as I like the insights page. I definitely recommend familiarizing yourself with this screen, um especially because this is where new relic is putting um a lot of their work into.

B

So in this case, we want to perform the same type of operation. um I can do, has ident and again we're going to see all of our cluster logs um and then to do something similar I can do, has client ip. Then we can take a look at that here. um You can add additional uh fields onto the right-hand side. If you you know, depending on what you're missing, but uh because these are fastly logs, we have a pretty substantial number of attributes, so you also notice on this left-hand side.

B

We can see all of those in case we wanted to add additional filtering, but just to highlight a few of those that are really helpful, of course, client ip and then another really good.

B

One is the last section, so we can take a look at laugh blocks, for example, so we can see that um you know the site has had 33 waft blocks uh since 12 hours ago, um so it can help you to review those um also, especially if your clients get um uh if they get a block through app um there's a id on here, uh request id you can use to. Actually you know review that. So if I wanted to take a look at, you know the laugh blocks.

B

So I can add that filter to my query. So now I'm finding all of my whack blocks um and then I can take a look at the request, ids and actually get some more details here. So um I believe, there's a waff block reason.

B

I'm not seeing it right off the top my head, but yeah you can actually you know, see the reason that the request was blocked. The other screens that I do want to show is this query your data screen. So this is actually the new relics, modernization of their insights. So um you know again one of the things that's really nice about heroic one is we're bringing all of our information in so um I'll. Just show you logging, but um you know in this case. Let's say we wanted to take a look at cache status.

B

um So uh if we just want to take a look at uh percentage of hits, for example, so we can see how many hits we're getting um over time, uh um but yes again with with new relic, you know you can add additional things here. So if you wanted to compare this against page views, you could uh plot multiple queries, so um I do want to show this uh query builder as well. This can allow us to do some more granular querying similar to what we're doing with insights.

B

So in this case, I brought over the page view query, but I'm going to copy and paste this query that I've borrowed from one of our other teams here and copy and paste this in here. So if I run this particular query, you can actually see our fastly hit ratio, so in this case we're querying uh the fastly logs um and finding a percentage where cash status equals hit of the of the entirety.

B

So you know you can use this to find out what your cash coverage is and then again, as matt noted, um we can then add these to dashboards. So, for example, let's say I want to make a fastly test dashboard, so I will add that there and then just for uh sanity's sake.

B

um Let's add a additional query. So um so again you know showing that we can mix multiple queries. Let's take a look and see how many exceptions we're getting so. Similarly, uh you know 90 35 exceptions. If I take that add that to my test dashboard, we get a nice little visualization, so um yeah. So I'll probably conclude it here, but uh once we load up the dashboard but you'll see you know, we can start to pull some really nice insights.

B

um You know into the relic and get a good understanding about the uh the health of our site, um and the only other thing I will note there is uh you know on that notice. You know new relic is very rapidly um upgrading this tool. One thing we noticed actually yesterday, I'm not sure when they launched it but they've started to add some um really interesting.

B

uh Parsing features, so um you know we'll be exploring that. Hopefully you know adding that to our our default rollouts for new relic in the future, as well as documenting around it. So um y'all can use that as well. So um with that, that's pretty much all I wanted to show on new relic, so um I think that's all we have for today. So I think we'll you know we'll uh we'll pause there for any questions y'all might have um about. You know either of these pieces of functionality.

B

So yeah, thank you for joining us and you know happy to take any questions.

B

B

I'm having issues loading the chat, so I'm not able to see if there's any questions in chat matt. Are you seeing any.

A

No, no questions.

A

Or feel free to reach out to us directly. That's also fine.

B

Yeah absolutely like matt said you know we're all in the cloud channel and the community engineering slack. So I mean especially if you're, watching this at a later points, the the the recorded session. You know please feel free to reach out to you know either matt or myself.

B

uh If you have any questions, so I think we just did have one pop-up in the chat matt. Would you mind reading that? For me yeah? Would you have magento at specific logs parsed to end new relics?

B

So not at this point, um you know right now we're using our syslog um uh our syslog protocol. So right now we're just getting um you know, basically the the the raw vlogs not doing any parsing on it. So um let me pop up my screen here. Like I said we, I literally noticed this yesterday I sent it to matt with kind of a like question mark yesterday, because I didn't notice this was there um yeah? You know I love nurella, because they're just they're they're they're so rapidly developing this tool.

B

But um when I noticed yesterday on the log screen is there is the ability to do custom parsing? So if I take a look over here, um we've got to manage parts where we can specify parsing rules. So I have not explored this yet, but this should be able to do that. So um you know in this case we should be able to do something.

B

You know where ident equals uh supportreport.log and then uh do some type of parsing so, um like I said, I'm I'm very eager to uh you know to to explore this, because you know when we, when we first launched it with new relic.

B

This was probably the the one thing that we were really missing them from having so, um but this this is, you know, custom per account, so you're certainly welcome to explore this on your own logs um and, like I said, I think that we will absolutely be looking into this more um and, if not rolling these out to accounts put out um documentation on you know what we think are best practices for parsing.

B

um You know there's, unfortunately, not a whole lot. You can do on. uh You know the the access logs, for example, without being able to parse out ip addresses or endpoints or us. You know status codes and things like that, so um yeah be on the lookout for dev docs, and things like that, where you know hopefully we'll be putting these out um and uh you know helping you all. You know get that parsing setup on the magento application logs.

B

All right, I think, uh borrowing any other questions. I think we're uh we're come to a wrap, so um you have any other any other news from you before we close the session.

C

um No, I guess not really. The only thing I wanted to mention that we released yesterday is a updated receipt tools and other packages.

C

So if you uh are interested to look at those uh just feel free to go through the release, notes and pick those up as soon as possible, but other than that we should be good.

B

Cool all right. Well, thank you so much everyone for your time today. Oh it looks like we got one more question about 2fa. um uh It should work with any git client, um that's authenticating over ssh. um Again. I think that you know the the main thing that you need to make sure of now is that you have uh the magento cloud cli installed locally, so that you will be able to. You know, generate your uh 2fa certificate, so um yeah any any get client that that works with cloud will be authenticating over ssh.

B

So um yeah just make sure that you're, using that, in tandem with the magento cloud.

B

B

Great all right well, thank you so much everyone for your time um and you know, like matt, said please reach out to us over uh slack. If you have any more questions.

A

Have a good one appreciate your.

B

A

All right take care everyone, bye.