Microsoft 365 Community PnP Webcasts, 24 Oct 2016

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: PnP Webcast - Sites Core and PowerShell Test Automation

Description

PnP web cast focusing on the PnP sites core and PowerShell test automation in which give more background on what kind of tests we do, why we do them and how the test engine works. You as a PnP contributor can use the test automation to follow up on how your test cases are doing by looking at the test results using our web site (http://aka.ms/sppnp-test) or via subscribing yourselves to a daily test result summary mail. Obviously, you can also reuse the code of both the test engine and test site for your internal projects.

A

Hello, everyone welcome to the speedy webcast today we're going to talk about being pico and powershell test automation. My name is beth johnson, I'm here inside Microsoft and with me I ffs I, you Vernon.

B

Hello, my name is Wes I wanna, I'm a senior program manager in shear bond engineering as well Thank, You, Vera.

A

Before we dive into the actual a test, automation content, a quick overview of what the SharePoint Batterson practices project is all about in in be a SharePoint pnp. We give you code samples, guidance, documentation, community calls case that is, and actually way more, like mzn articles, etc. A lot of content for you to reuse and to start from their own from that content is kind of categorized in the number of teams.

A

Nowadays, we have a lot about the SharePoint framework, because that's the new thing which is upcoming, but there's also tons of content around SharePoint, add-ins max grab over 65 api's and a broader context, also about provisioning and any client site development. So if you want to know more about sharepoint pnp one address to remember, it should be a k dot. Ms / SP, p and p.

A

Alright, so Oh going back test automation. What is it? Why did we do it? So the pnp project has been growing over the years and one of the the two main components of P&P that we actually I think fest. It was 2015 that we actually really added those and made them and they became popular. That's the decor library and pnp powershell, and these discs or library and partial something are really pop, lock components used by many many people inside the community.

A

So actually we have thousands of talents using those on a monthly basis, generating millions of requests on a monthly basis, which means, since this broadly used, we really want to do our best to keep quality a high of those components, and for that we set up our a bit test automation.

A

So essentially, we are automatically testing those components across different environments on a daily basis to ensure that keeps on working that we detect bugs early on so you'll, see later on the slides that with example, test against what we call our Edo confinement, which essentially is the first tenant in the food chain of deployment. So we deploy first, he talked then deployed the other environments. Eventually, we deployed the your environment.

A

So if we test against that very early environment being a doc and we can detect problems, we can also get them fixed before they even hit your your tenant. So, overall, this destination will not just help get the pin p core component stable end of the right quality, but it also will help microsoft to prevent issues from being rolled out.

B

And maybe on that one bird, just a few notes on because it adds the real story and real life experiences on that one we've been able to catch quite a few actually box on the a dog and a box in the e docs are absolutely understandable, because the doc is our development. The day-to-day build system for sharepoint online. With this test automation, we've been able to figure out whenever there's the new builds coming out.

B

If there's any random box, which would have been back there than the actual production tenants of the customers by using a well and we were able to catch them and fix them before they go live for the customers, it's.

A

Absolutely and reason why we can able to catch issues. You could argue like why don't the developer, sketch these issues but being be testing, is you can see? This is technically speaking. It's integration testing, we test against life environments, whereas the typical developer testing is done against the more developed, smaller developer environments and with life of ironmans. You have things like caching, which play a big role and we've seen several issues we caught several issues that were due to a gashing goes due to this.

A

Sheer size of the environments doesn't mean a thing works on if it works when your developer box that it also works in a huge environment. So that sense. Yes, we've been able to catch many issues already, and hopefully we don't catch future issues with reality learns that developers make mistakes, so we will keep on catching issues. I.

B

Yeah I think that's part of the development cycle of any product. Yes,.

A

B

A

Not that Mike's of this bad development and not saying that definitely not at all, but it's either general thing in any development project goal. Vs developers all understand and agree with that, setting.

B

Sure sure, and at one point that's just the kind of a you mentioned birth mention that one already but really the B&B is more run scenarios and functionalities, and that's really the key advantage while will and why we use the BMP also for engineering testing, because then we know that customers are using. Thousands of customers are using BMP core component and powershell, and they are scenarios their functionalities and quite often they're. Not just individual method calls. They actually run through series of functionalities, to enable, let's say a printed sides with B&B.

B

We can be and be that automation we can catch those things in this series of of scenarios, which is a really great advantage. Yep. This.

A

Is actually a very good point at you race, so we tested our flow, a scenario which compromises actually maybe 10 15 see some goals is the the technical testing done by the developers is more focused on okay. I have this particular method. This api at desis api. It works, but we tested it in a in a flow, and that sometimes means the api isn't working anymore, yeah sure.

A

So that's about for an introduction on what it is. Now us, big community member, you can get insights in our test results, so we have a site showing you. Our daily test runs, so you can go over there and have a look and we'll show you in the demo later on. You can also get a daily mail digest listing all the test results so did and the reason why we actually share this with you is community. It's like if you're invested in NP and be using it.

A

You want to stay up to date on how is behaving how it is going on, so you can via the site and the Daily Mail. Also, if you created a new functionality, let's say you added new PowerShell command lets and you create a test cases for them. I think you're interested also in an understanding and knowing whether those test cases actually work or fail, and if they fail, you want to understand why they fail.

A

Maybe you need to change your functionality, maybe there's something that needs optimized, so you can learn from the test results and then, if needed, change, your contributions, kind of improve them, etc.

A

Okay, so how do we run those tests against the co library or PowerShell? So we have a set of service where I think. Today we have for service with sharepoint 2013 box, sharepoint, 116 box and two independent build service.

A

Essentially what they do is very simple: they use an MS build script which pulls down the data from get up, so we take that the death range pull it down on on the box, and then we build it run the unit tests and store the data in a central sequel, asier database, and that's something we do for sharepoint 2013 for empty. So the regular tenants that you guys are using d dock is our with your our early environment for deviant next four dedicated customers.

A

Then we have a dock, which is the early environment for for the regular, empty customers and cannery is something like environment in between between eat, okay and then empty, so that that's going on all on a daily basis. Now, on top of that, on top of that sequel, a geodatabase, we have an edgy web app that you can use to get access to the test results that web app is mostly accessible, so you can just go over there to akms /s VPN p dash test, and you can see the results from the public environments.

A

So all test results coming from mt from sharepoint 2013 sharepoint 116. They are visible for you. Anonymous way results from a doc d, doc and cannery. Since these environments are not publicly available, we also can't make the test results public. So you won't see those you might wonder. Why do I need to authenticate? What's the dedicated users doing so delicate that users they can subscribe to and a daily email? So remember that I told you it. You can have like a daily email with the test results in order to get that mail.

A

We need to sign you up. So we know your email address and your verified that you really want to receive that meal, that we not spamming you. So that's why you need to authenticate at that point. The second dimension authentication is core team members.

A

They can see it up canary and indeed oak results, and so we check on one who is authenticated to show the right data.

A

This is a different representation of the intestine environments, which already explained in a previous slide. Essentially feature team is building functionality and they release on on a very agile base to the a moody to the actually the e dog test environment, meaning when I talk about agile, it's really now erase on developer, commits a few hours later. The code can already be deployed to eat okay, so it really really agile, and in there we have two stars. We just eat oak and also deduct for the dedicated environment.

A

So there really early early on test environments, then typically, a microsoft itself does some dogfooding. So we, the microsoft production environment, gets updated first and we use it for a while. Then changes go to cannery, which is an early environment for some some ice fees and partners.

A

Finally, it goes to a first lease and global liveability so and the stars indicate in which validation rings. So we actually do testing today.

A

Test passed balls.

A

Silly dog in this light, I'm going to explain you. How did the test engine actually works? So we're not using vuze to do a test automation to run the tests? We have our own lightweight a test engine which essentially is a console application which uses an irregular standard. The MS build file some custom-built tasks. So you might wonder why do we need custom, build tasks?

A

And the reason for that one is that when we do the PNP go testing of our shop testing and for people who are familiar with that and we actually create a app config file with input data for the test. So those test cases depend on the outcome to file being present with a custom-built task. We can generate the right app config file for the right environment, because these same tests are running against a doc and against empty against sharepoint 2013, sharepoint 116.

A

So those two components- the bill file, custom-built tasks are used by M is built so that the console application I kicks off Emma's built that then uses the build file to pull down data from you get create the app config file, compile the downloaded solutions so Evan. Once everything is compiled, it kicks off the visual studio tested constable XE, which is actually the visual studio test runner, but then from a console it kicks that guy off that guy starts to run.

A

It takes the first test and then we use one little custom component diversity do custom test logger over here, which essentially is a customization in velocity estimation which, where you can get the test results. So when this guy is running a test once the test is done, it calls our custom components which you then use to persist. The test result in sequel, a geodatabase and not on regular file system like the default deja vu studios test automation.

A

So we should be sharing this with you and the code as well, because it is pretty handy and lightweight. So if you want to do some test automation in your company, you can, of course we use this one. That's.

B

A no doubt that one option so about a again we do and use whatever the it was your preference, but to be fair, there's no harm for us showing how to how those things are done. So, if you want to build an exactly the same setup, absolutely make sense. Yep.

A

Yep, that's us why we share it to sharing is caring, your front, pin it all right. So, what's on the radar far for this test, automation first word is adding additional test cases for the provisioning engine. There's really a lot of test cases in there and maybe vested as a topic for a future webcast to talk a bit more about how this test cases work and how community can extend those test cases and grow them, but essentially we're still adding some water test cases. So we can be better prepared for the upcoming engine.

A

Refactoring we're also looking into automatically running the test case from the pjs core library as part of our destination, we're just not not yet there today but I on the radar as well and finally we're looking into to using a selenium, which is an open-source test framework to do coded UI test. So essentially it would mean that'd be good, for example, pull down the partner back, build it auto, deploy it and test it. That would be the end result of this operation.

A

So I can really also add some some UI testing and testing of our reference solutions. All of this to show that they keep on working with evolution of sharepoint, and so we can keep giving you guys high quality, yeah.

B

No, no, no selenium and obviously on a coded UI test. Those are slightly difficult kind of a challenge thing to do, because we are evolving sharepoint online, so fast. So obviously we can do automation on B&B, you guys like the BMP partner back or if you have your own, let's say application, remote provisioning application or what or whatever application when in sharepoint online. If you have a dependency on Dom structure and that might be change tomorrow and that's always a small chance on a codec UI tests, yep.

A

This may be also one of the reasons why we want to do this code. You write this to detect. Maybe the septal subtle changes that happen. So we can adjust our samples where needed sure.

A

Okay, Oh, quick demo, let's go over here, so I went to this address, which is also the AKMs address. If you use that point you end up over here, but this essentially is test automation about sharepoint, P&P, calm.

A

When you go to the home, you will see kind of a dash both giving you the latest test results. So so what are? They have.

B

A shows here as an example, so yep.

A

Yeah we don't have issues. Is this it's actually. We were lucky that we see three different icons in one screenshot, so that's good, so I kicked off. Some test runs before the call, and you see that that these guys are done because they are a check mark. So the tests finalized. You see the type of test, the credentials test or an appt only test. So we test both combinations of authentication models. You see the environment sharepoint 2013 2016 online.

A

Here you see this. This guy is still running. This guy is getting started, so it flips to running within a few minutes, and what else do we have in this dashboard? So if you go to scroll more to the right side, you see a number of test cases, how many past skipped filled and the lock. So what's typical scenario for how do I use the tool in the morning? When I look at this, I see like okay. There is we have this case powershell. We have a two failed test cases in the power Sheldon.

A

So let's click on the field. One brings you to a page showing the failed test cases together with the skip test cases they're not found in the past test cases, so we can search for a test case here or if you want to find particular one, but typically you're interested in the failed ones.

A

So you can see over here. You see the first error when you click on actual test case. You get more details. Actually our message: the stack trace the output of the console. All the data is there for you to understand why it failed and, if needed, to try to fix.

B

It so, in this case Burt what would be the reason so that had a failed to for PowerShell as an example, oh, that was for appt only yeah, so.

A

Yeah it's in and authorized error, so we need to investigate it. That's arrows are not always indicating what really went wrong with ya with this app only here. So somehow this get bumped up. Web templates test failed for partial okay. Also PowerShell is fairly new in the test suite, so we added it like a cup of weeks ago, which means that that's not everything is ironed out. Yet there are some test cases which keep on failing.

A

So we still need to do some work to kind of make sure that we're the core component testing is already mature. If test cases fell over there, then yet something's wrong all right. One last little thing to show you guys, so if I go to settings and I will be forced to sign in by sheer it. This sign in.

B

A

Now and then, once you're signed in, you will see your status. So typically you will be community member status, but you can important thing is that you can check this set me a daily test summary enter email address and update that point. We will send you a mail to this address to confirm your subscription. If you, then by the mail, confirm subscription, you will get a daily status meal, showing the test results.

A

All rights I think that covers up the demo. Let's flip back to slides sounds.

B

A

Just to recap: what are the falkland resources for you, our destination site, the one shown in the demo, so https destination, don't share one p and p dot com, and if you want to leverage the source code either from the engine which we explained on slider given, but it's like with the bill. Let us build script to custom-build tasks. So if you want to reuse the engine for your needs and engine currently resides in the side, score pas story dislocation certain size called as a tools folder containing the dis unit. That's the tool.

A

We also recently shared the actual test you I, so the website you saw before and the engine that sends out smell, ification I, don't expect you to typically you just reuse it as this, and but it can be an inspiration for you to use your company. So that's why we just shared it. So you can hear if you want wine average its. Why not yeah, absolutely yeah I think that that kind of summarizes the test.

A

Automation webcast, so things to remember is that we do it estimation to keep improving quality and keep up with the evolution of the ship on platforms and for you, you can always see how pimpy is doing, and if you build your own test cases, you can see how they are doing as well. You can get kind of a daily mail, so you don't have to go to the site each day and see like how did my test case run now.

A

You just described you get your daily mail and you see email box like everything's, fine, Ahnold, fine, because.

B

Everybody wants to have more email, but yeah.

A

B

Their preference for peasants, just out of curiosity, were to add what was the number of feature feature: I, sorry, integration tests, what we actually have, because I can't remember that it's.

A

Getting close to 400 against.

B

Well, it in the whole point is that they are not unique tests. As such, there scenario testing.

A

They are scenario and / integration, testing, yahoo, the tested always tests against life environment and we need were typically testing a particular flow, a particular scenario, because the pin pico component ap ice and extension methods. These are implementing a particular scenario. Some snails are small, some are bigger, but yeah. So this is kind of scenario. Testing and same applies. A power saw.

B

A

All right, thank you, guys and hope to see you soon in our next webcast. Oh, why thank you.

B