Microsoft 365 Community Microsoft Teams, 16 Mar 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Utilizing Mobile First Device Capabilities for Microsoft Teams Apps

Description

In this 20-minute developer-focused demo, Microsoft presenters Anuj Bahl and Shubham Chauhan introduce viewers to four device capabilities (Image, Location, Mic, and Scanner APIs) now in Teams Client JS SDK that developers can utilize for extending bots, tabs, messaging extensions and personal apps, in mobile apps. Reviewed mobile device capabilities Roadmap, 3 UI demos, and scenarios with API configurations. Have a look at the mobile People Picker Control in development. API documentation with code snippets is current. This PnP Community demo is taken from the Microsoft Teams community call – March 2021, recorded on March 16, 2021.

Demo Presenters:
• Anuj Bahl (Microsoft)
• Shubham Chauhan (Microsoft)

Supporting documents:
• BlogPost - Four device capabilities developers can utilize for Microsoft Teams mobile apps | https://aka.ms/devicecapabilities
• Documentation - Build apps for Microsoft Teams | https://aka.ms/TeamsDevDocs
• Documentation - Device capabilities | https://docs.microsoft.com/en-us/microsoftteams/platform/concepts/device-capabilities/device-capabilities-overview?view=msteams-client-js-latest
• Documentation - Request device permissions for your Microsoft Teams app | https://docs.microsoft.com/en-us/microsoftteams/platform/concepts/device-capabilities/native-device-permissions?view=msteams-client-js-latest&tabs=desktop
• Documentation - Integrate media capabilities | https://docs.microsoft.com/en-us/microsoftteams/platform/concepts/device-capabilities/mobile-camera-image-permissions?view=msteams-client-js-latest
• Documentation - Integrate QR or barcode scanner capability | https://docs.microsoft.com/en-us/microsoftteams/platform/concepts/device-capabilities/qr-barcode-scanner-capability?view=msteams-client-js-latest
• Documentation - Integrate location capabilities | https://docs.microsoft.com/en-us/microsoftteams/platform/concepts/device-capabilities/location-capability
• Documentation - Microsoft Teams JavaScript client SDK | https://docs.microsoft.com/en-us/javascript/api/overview/msteams-client?view=msteams-client-js-latest

Stay connected:
• Twitter https://twitter.com/microsoft365dev
• YouTube https://aka.ms/M365DevYouTube
• Blogs https://aka.ms/M365DevBlog

A

I'm anuj bahil and I work as a program manager on teams mobile platform in the last six months, or so we've been investing heavily on mobile-first innovation and capabilities. We're not just investing on your features for mobile users, but also making sure that the platform constructs are easier to use and apps are more discoverable and simpler to acquire on mobile.

A

Today we are going to specifically talk about device capabilities, and- and these are essentially hardware capabilities that come on your phone. uh We want to showcase the various scenarios or to you uh on how extensible app developers can utilize these capabilities to build richer mobile first applications.

A

I want to briefly talk about the various platform constructs um and how they essentially look on mobile. I'm sure most of you would be uh familiar with this, but we have bots, which are conversational entities that perform a certain specific set of tasks. So bots can register with the ms bot framework and can and then act as essentially any other member as part of a chat or a team, and bots can communicate with users respond to their questions, proactively, notify them about changes, etc.

A

Then we have tabs. These are canvases that can be used to share content, whole conversations and third party services, and these are essentially nothing but web views that load a website. Url messaging extensions is a construct that users can use to aid themselves, while they're composing a message and on mobile it sort of comes up when you open up the the compose extension tray and, lastly, we have personal apps, and these are similar to tabs, which just exists in the personal scope.

A

So while you have tabs that can be configured and designed to be more collaborative in nature, as they can be pinned to a particular chat or channel users can use personal apps to increase their own productivity and that's what comes up when you swipe up or on the home page in the in the more menu all right. So before I hand over to shubham, I just want to talk uh a bit about um why we're really investing heavily in mobile first capabilities before the pandemic.

A

The usage of teams is sort of restricted to north america, but in the last year, or so, we've seen the usage grow in other parts of the world, uh students, front line workers and other blue collared workers that don't have access to large screen devices use mobile on an everyday basis and mobile. Only users today account for close to 20 of our teams. Users, and this is one of the fastest growing user segments, and we do expect this growth sort of continue in the near future as well.

A

So, hence we're like focusing on empowering teams developers to build create mobile first apps by enabling device capabilities in the in the js sdk uh over to use shubham uh to talk more about the different apis. We've exposed.

B

Thanks thanks uh here, one uh work with android on the platform, uh so I'll just jump right into the device capabilities which we have developed over the past couple of months, as well as we'll also look into some capabilities that we are currently working and will be releasing in the future. uh So here's a quick road map of how things have been over the past and how it looks in the future.

B

We started off uh with the image capture and you know photo gallery uh image, picking kind of scenarios first, so the first two there, uh the first two items here on the timeline represent the same. uh We have used office lens, which is like a powerful tool that microsoft leverages to power scenarios around capturing images through a camera. Then the third one here is recording audio using a microphone. This again is a kind of a very common uh scenario.

B

If you talk about mobile, especially for mobile first scenarios and then the fourth one here is a location detection interface, which basically means, uh if you're familiar with, say how we kind of share our location uh for, while, while booking a ride on uber or lyft, so that kind of map interface can directly be leveraged using the aps that we have recently introduced, um then the fifth one here is around barcode: qr code scanner, again a very mobile first kind of scenario, uh so we again use office lens sdk to power.

B

Our uh control here we'll go into details uh of this, I'm just quickly going through the timeline for now, uh then uh till now we have only talked about the mobile capabilities, but some of these capabilities uh we thought, would also make sense to have the desktop counter counterparts supported as well. So then we kind of work to gain parity uh in terms of functionality on desktop. So around this quarter we have been working on uh the first two set of capabilities, around images, camera location and making sure that it also works in desktop.

B

So this is our current work that we are doing then. At the end, the last two are around a people: picker hvc hvc, meaning a high value control. uh So there are very common scenarios in which you would want. You know a control that allows an end user to select a user say from a chat or channel or within the org, in order to say, assign a particular task to a user or just add the user in a particular workflow.

B

We believe that's a very common workflow, and we also wanted to make sure that this kind of control is added to the client sdk itself, so that the developers can directly leverage it. Instead, you know using their own uh putting in their own effort and building such a control that can be easily leveraged directly. Then the last one is around video capture capability, which we'll be picking up sometime next quarter, I'll be going into quick demos of how these capabilities currently work. uh So the first one here is around um image apis.

B

uh So here we just wanted to demonstrate uh the power that office lens provides uh so june. If you can just play the demo uh I'll, just uh this doesn't have an um voiceover. So I'll just uh walk. You into what's happening so here we just created a sample app to demonstrate the power that uh office less provides uh in terms of the edit functionalities and kind of high high level.

B

uh You know image capabilities so, for example, the scenario here is this: an app a parent is trying to share the image of his or her child working or with the classroom in a particular app.

B

So, as you saw here, uh the app could directly, uh you know access certain functionalities which are natively available and the parent is able to either click an image from camera or just select one from gallery and then office lens provides a host of functionalities such as you know, you can add a text annotations, you can add filters, you can add free uh free hand, drawing on drop of the images.

B

Basically, you can have all the functionalities which everyone is kind of very familiar with now, and that can be added to the image before sharing it right back to the underlying web app. So this is more about it. uh Then um it's kind of standard stuff. We can. We can pause this demo. uh It's just submitting the image to the web app and uh move to the next one, so this demo was around demonstrating how uh powerful office lenses in terms of high level edit functionalities over images. uh The next demo.

B

We have that's a common uh demo. That kind of demonstrates three of the capabilities, including the one that we talked about, uh which is image and then location and mic. So this one has a voiceover so I'll, just mute myself for a minute or so yeah june. You can just play the demo with.

C

The help of number of device capabilities now enabled on teams platform contoso, has enabled a tab app that lets field agents record information easily on teams itself that is instantly viewable on the contoso repair channel. The field agent can quickly create a new inspection item and raise the ticket.

C

The feed agent can use the attach picture option to capture images of the car using office lens. They have the option of choosing these images from the gallery or clicking them with the camera.

C

Using the multiple image support, the field agent can upload multiple images of the car. At the same time, they also have the option to choose and pick the current location of the customer by dropping a pin at the map interface instead of typing out a description of the car. The field agent can record an audio description of the car itself on recording the audio successfully.

C

The feed agent can submit the ticket and, finally, the information is added to the tab app in the channel.

B

Yep, uh so that was a short demo of a simple app. We built that leverages three of our capabilities around image, location and audio or mic. The last demo that we have at least for this area, is around a qr barcode scanner capability. We just went live uh last month, uh so yeah this again has a voice over so I'll mute myself june, uh you can play the uh demo. Here's a demo showcasing qr and barcode scanner capability, that's being added to the team's client sdk.

B

Max a contoso employee needs it support to fix his laptop. He uses the incident reporting app to raise support, request.

B

Earlier, entering asset code manually used to be really inconvenient, with the code being lengthy and typed, usually in small font. Also, there was always a risk of making mistakes now max uses his phone's camera to scan the asset barcode which automatically fills in the asset info fields.

B

This makes the process of raising support ticket much simpler and after entering the issue, description max just hits submit to generate a fresh support. Ticket max's life is easier now, thanks to qr and barcode scanner capabilities present in the team's client sdk, so that was around the qr barcode scanner capability, which just went live around last month.

B

So so this is a control that we are currently working on. um So this is again something that we have been hearing a lot from our developer community and especially our 3p partners, who all of them had had a requirement. A control which can be used leveraged to you know, pick people uh it's it's not an easy process to do, and if, if something like, this can be extended right out of the team strand sdk, then that would add a lot of value to the developer community.

B

In general, so we listen to that feedback and yeah. He is his uh kind of the I work that we are currently doing, and uh these are just a box for now, but it's under progress. So, as you see, uh here's a sample, a purchase, approval kind of uh workflow wherein one of the uh steps is to select the approver. So the second and the three third screens that you see are here. These represent the experience that would again uh be coming in in form of an api in decline. Sdk.

B

So this entirely would be just one epa call away, so developers can just leverage that call uh that capability and you'll have you don't have to invest as much in building a people picker on your own. So here on the second and third screens, uh the end user selects the respective approver and, as you see in the last screen, uh the name of the approver and relevant details get shared back with the underlying app and the end user hits send and the workflow gets completed yep.

B

So so these were the demos uh now coming to what api is to leverage. So uh I'm glad to tell that all our documentations have now been updated and published.

B

uh I will provide the link to all everything that we just demonstrated all the apis and how they work and uh even uh sample code snippets to make sure that uh you don't get stuck anywhere so I'll just quickly go through the apis, uh the corresponding apis uh to the relevant uh api uh capabilities we just discussed so the first one uh is around images, uh image capture and also we have another api for uh providing an image.

B

Viewer uh control as well so select media is the api uh that you will need to uh invoke or call to kind of capture uh or let users capture and attach images to the uh underlying web. So uh so I just wanted to cover the configurations, also really quick.

B

uh So we kind of understood that there are different requirements or use cases coming in uh from uh from mobile-first scenarios such as, for example, there was a scenario which one of our customers was asking that you know we want to create a check-in kind of app wherein the user is restricted to capture image. Only using the front camera. We don't want users to use rotate the camera at all, or there were scenarios like you know this.

B

We have the scenario that we want users to scan a picture instead of capturing a picture, and then there were scenarios around. You know uh in in um quality, check or quality uh inspection kind of scenarios. They wanted to uh do a free hand annotation over the image before submitting it to the app so different scenarios uh kind of translate here to different configurations that we have kind of provided.

B

So you can change this configurations as per your need of the app and respectively, the uh control would make sure that it opens in a relevant configuration, so the first second one here is a number of medias that can be attached pretty straightforward. Third, one is the source source means uh whether you like to let users capture an image, or you know pick an image from the gallery or you know let the users choose which one they want to do uh then. The fourth one here is the start mode.

B

So um office lens supports a couple of uh scan modes as well, instead of like standard uh image capture kind of mode. So it supports document scanning whiteboard scanning and business code scanning. So, if you want, if you have a scenario wherein the control should open in a particular scan mode, then that too is available in form of configuration, then the fourth last one is around cameras which are, if you want to default, to front or back camera or want to restrict users to use only a particular camera.

B

Then this is the configuration to use uh the last three are around uh editing, so inking applying text stickers and applying filters. So these are just boolean uh boolean configurations. If you set them to true uh these capabilities, which will just show up in the uh camera control, uh then the last thing I want to cover is around a view, images view.

B

Image is another api that we have added to the sdk, which can directly directly be leveraged uh to you know, have a image, viewer kind of control so that you can just use an inbuilt control instead of building your own image viewers to preview the attached images, so the coming to the location, uh apis, uh there's just two pretty straightforward apis here one is get location, one is show location.

B

So get location is you would be used to capture users location? uh We have configurations which kind of let the developers decide whether they want to show a map or not show a map, and if they want to show a map, then would the user be allowed to move around the map and pick any location on the map, or are they restricted to submit they only their current location?

B

So all of them are the configurations that can be created by using allow choose, location and show map variables, which are the configurations that we currently support, so allow choose location would mean you know, a map would open. uh This again is coming from a mobile first scenarios that we have heard a lot. For example, in some scenarios the apps just want to show the map as a confirmation screen, and they want the user to submit only the current exact location for verification kind of scenarios.

B

So listening to those kind of feedback, we have this configuration that restricts our user to either submit the current location only or maybe enables users to pick any location on the map and share with the underlying web. App and show map is, is just a com like, as the name suggests it it's a configuration, if set to true it would show them up if set to false. Then this is kind of a scenario which is also currently enabled by the html5 uh get current position kind of api, so it just uh kind of fetches.

B

The current coordinates from the gps of the device and shares with the underlying web app without showing any particular map on the ui, then the other api show location. It would just open a particular location uh that you provide to this api on on the uh native mobile, supporting app so coming to the last capabilities that we have rolled out for now, a microphone is again very simple.

B

There's just one api which you have already discussed so select media is kind of an umbrella uh api, so we have just extended, select media, which we discussed earlier to also support my capability. So, basically, the interface is same as as seen in the demo using this api, a small kind of bar would appear at the bottom of the screen, wherein you'll the end user will have the option to stop the recording whenever the user feels like ending the recording and the recorded audio stream gets shared with the underlying gap.

B

So the only configuration here is max duration, which kind of is limited currently at 10 minutes, and if you set this duration to say one minute, then the recording would automatically terminate after that set period, then the last capabilities around barcode, qr code scanner, again just one api, which is called uh scan barcode. The only configuration we have is timeout intervals in second, it's currently marked at 60 second and default uh is 30 seconds, but it can be configured for the needs of of the end user.

B

uh All the experiences were kind of demonstrated back uh in the demos so yep. I hope everyone can relate which apis correspond to which capability and we just post, we just shared a blog post recently on all the four capabilities we just talked about, which is to summarize, which is around image apis, which is capturing an image or picking an image from gallery or viewing image then. Secondly, around location, which is uh getting a user's location or showing a location on a map.

B

Third is around audio or mic, which is recording audio on using the device microphone and the final one was around qr barcode scanner, which is again uh being added being added to the client sdk as a ready to use control right out of the box. uh So you can just check out this blog post. uh The link is ak dot ms device capabilities and also uh respective documentations.

B

So that's all we had from the device capabilities area. uh I hope everyone did find some value in what we have been developing from, especially uh looking at the mobile first vision that the team currently has do share our feedback on the chat and yep also share any questions on the chat. If you have any thanks, what do you choose.

D

Awesome, thank you chevron. Thank you anders, and the ship mentioned on that blog. You can find all of the the resources um and the documentation um that they mentioned as well, so be sure to check out that link I'll. Let me go back so that you guys can have it again in front of you if you haven't copied it already, but aka dot. Ms slash device capabilities.

D

Thank you guys. So much.