South Big Data Hub Education & Workforce Working Group, 5 Feb 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Education & Workforce WG: Keeping Academic Data Science on Track

Description

Date: 02/05/21
Presenter: Karl Schmitt
Institution: Trinity Christian College
Title: "Keeping Academic Data Science on Track"

http://sbdh-prod.ideas.gatech.edu/resources/newsblog/education-and-workforce-working-group

A

So uh carl schmidt uh he's going to be talking about excess and assessment, keeping academic data science on track. So you want to introduce yourself carl and take it away.

A

Oh you're, on mute still.

B

Okay, here we go okay, I had to find our place to unmute myself, hi hi, I'm carl schmidt, I'm currently at trinity christian college. I started there in the fall and they hired me to do program, coordination and design for a new data analytics program there. I've also been at valparaiso university, hi john.

B

I saw you logged in and designed the program there and been working as part of the acm data science task force on um curriculum guidelines from the computing side, so I've been engaged in a lot of sort of high level things and some of the stuff that renata's organized and some other hubs have organized um related to this, and so today my hope is to sort of point you in the directions of where you can pull the things that you might want to have to assess both for programming, or course, level stuff um and a little bit less on the.

B

How, besides giving you some pointers about stuff that, I think is really important to make sure we're thinking about as we're implementing these programs and courses so that we are continuing to keep the diversity that's already existing in our classes, successful. So with that, I will go in so to summarize, for anything, if you, if you stop paying attention, pay attention to this slide, and then you can, you can tune in and out a little bit more.

B

uh Here are your two sort of quick takeaways um right now, the most comprehensive set of of competencies or learning outcomes and stuff is still the edison data science comp or the edison project's competency framework for data science. um This is something that came out of the eu in 2017 and um I'm still referring to it as the best most comprehensive thing.

B

This the fact that there's been a couple of things published since then, um as far as I can tell nothing else, seems to have the comprehensibility of this, although there are some things that are coming out now, that are sort of addressing some of these pieces and sort of that follow-up is, if you're, only looking at computing competencies. So this does not include math and stat issues and and minimally contains domain stuff. The acm guidelines is more complete than edison and a lot more detailed, um but you can't use it for every single thing.

B

The other piece is thinking about the sort of how you actually go about doing assessment, and so I was really happy to hear hunters talking about doing sort of this mastery-based idea and making sure that um you know there's a there's, a level of competency, that's aimed for um one of the challenges that a lot of courses can run into, especially as you start getting lots and lots of students is comparing relative to other students, rather than actually comparing a student to themselves or against what your learning goals are and there's tons of research out there and I'll point.

B

You to some of those places that say those are stuff that really disenfranchises people that are perhaps coming from a slightly weaker background. So the fact that they've got a mastery-based model means that those c plus students coming in are not going to be dis advantaged. Besides the general knowledge that they don't have in terms of the assessment, uh as opposed to simply saying oh well, they didn't achieve things. Well, it's not being assessed against a cheat, it's not being a sense against the other students that did achieve things.

B

That's when you guess obsessed against did they meet the goals that we want them to take all right. So that's the quick takeaways, here's a little bit more, a dive in. So, if you're not aware, there's about counties there's about six uh curricular level, guidelines that are in existence in publication that you can go find someplace.

B

um The edison project is probably the earliest one. That's out there and most complete uh lots of people refer to the park city report, um and I have that wrong. That citation wrote is d-vox at all um there's also some public there's also a public at least one. I think. There's actually multiple now, but I only reference one that are sort of assessing deployed programs, there's one from the business education side. That is still talking about data science and analytics.

B

But it's it's a much higher level thing and the the particulars about it are a little bit rough to get out more than just sort of the really high level and there's a an early start for a group from that's calling themselves the initiative for data science and analytics or yeah initiative for data science and analytics, um and this is meant to be sort of standardizing some of the professional level assessment stuff.

B

uh That paper came out in harvard data science review and it right now it's at the high level, although presumably it'll continue to come out and then the last is uh acm just approved a week or two ago the the education board approved the final draft of the computing competencies for undergraduate data. Science curriculum um I've linked here where the task force website is, although I just checked, and this morning we still don't have the actual final draft up.

B

We still have second, the second draft, which is close, but we've we've definitely made some improvements since then, um so keep your eyes peeled. I know that we've got a talk at uh the symposium for data science and statistics in june, and we've got a talk at sixty, so the computer science education conference coming up in march um both announcing and sort of presenting some of the stuff from there. I'm not going to talk about that in detail. Come to those talks.

B

If you want to hear more detail about the curriculum guidelines, um but they are out there they're there and if you're familiar with the cs, 2013 they're at that level of detail, so they're much more detailed in terms of what you can pull out for particular courses or even thinking about high-level stuff. But again I'll put the caveat. It's only for computing things. Please do not take offense all you mathematicians and statisticians. We know we didn't cover math and stats. It's coming we're trying to get everyone on board for it.

B

So why am I saying? Edison is sort of the place to look at so as part of a nsf brand.

B

I've been, I did a literature analysis um sort of looking at these different curriculum guidelines and actually comparing them to edison as the baseline and the early thing to sort of see, what's changed and come out and um edison is the one of the few that's got published out and has now have like actual curriculum designed based on that across the eu, and you can see here that um a lot of the other early things from the u.s centric publications were not getting nearly as much coverage, and um so what I've shown here is uh kryptendore's alpha.

B

So treating all of these curriculums as sort of an idea of a coder of what topics are here: there's not a ton of agreement.

B

um At the end of the day, if you actually pull out the topics, there are 15 topics that all the curriculums agree to be included in and if I listed those out, you wouldn't be surprised at any of them.

B

It'd be like basic data mining and, like you, need to learn unsupervised, machine learning or supervise machine learning, and things like this so they're not there, because I didn't want to like waste people's time with talking about those, although I'm happy to share some of those details, if you want a little bit more interesting is the sort of four things that all of these curriculums don't include and really there's five that everything doesn't include, um and I think this is to me there's sort of the two partic the most interesting one.

B

Is this one, this open data and I think that's an artifact of the us-centric curriculums versus the european centric edison project, but it's sort of interesting to say that none of the other things included anything about this okay and then um sort of recognizing the issue about domain. The addison actually has so the main knowledge directly embedded in their curriculum competencies, and most of these other curriculums don't have any explicit domain things included. So that's the econometrics, the data, different marketing technologies and the operations.

B

Research, I think, is a again that's a terminology, a little bit that edison explicitly included the only one that included it was the initiative, the iad iadss, all right.

B

Moving on, so I I've also had some information that sort of looks at this same level of um questions for um from practitioners, and this is particularly practitioners and faculty that are teaching this, and so this is a little bit more aggregated because of the survey method that we used.

B

But you can kind of see here that compared there's still a lot of agreement in uh in the data analytics topics and the data management topics, sort of pointing to the fact that there is a possibility of identifying the things that you should be assessing at the program level or in some of your courses.

B

That sort of the community seems to have reached a consensus on these are topics that should be in a program no matter what um and the rest of the things are sort of still up in for debate, and so this is sort of answering this question of like what should you be assessing taking a look at some of these things from addison and those 15 topics is a good place to target yourself for saying is my program covering the things that a data analytics program should be covering to meet the needs of what's coming out for practitioners what's coming out for um sort of across the place, so that you know you know your major is correctly prepared to apply for all those jobs that say, data analytics or data science, all right, so I didn't I did so like hunter, I'm not tracking my time particularly carefully, so I'm gonna assume that I'm yeah, I'm gonna, assume that I'm almost out of time.

B

This is why I, I know I talk quite a bit on that, so I wanted to point you to places that I have been using for myself to learn about uh effective diversity, recogni, diverse diversity, inclusive assessment and things like this, and so the two biggest places that I can say that there's tons of co of collected research.

B

So this is not me being like hey or any. These are the one one or two people that do it, but both nc wit and the american association of university women have fantastic collections of resources that talk about all the issues and if you're only going to go, read one thing about this: go read that aauw solving the equation.

B

uh That is a nice comprehensive report that talks specifically about women in stem and with a strong emphasis on um computing related things, and they go from everywhere from getting women in and ethnic diversity into your program, to making sure that they succeed in your program to making sure that they get enough professional development so that they actually get jobs and continue to expand the women in the workforce in those disciplines.

B

um There's also other good stuff out there. These are just the two places that I think are the most concentrated that I like to send people to.

B

First, if you start getting engaged in this and reading this stuff, you'll see you'll you'll get connected to other people that are doing good work in this um and sort of here's, some the sort of big key takeaway ideas that I take out from reading this literature about the process by going about to assess these, which is setting competency levels um to me- and I said this later right rubrics- are great- that's a great way to assess a competency level and make sure that you're being fair in your assessment rather than just sort of arbitrary and how you're doing it um another good technique that sometimes is more challenging, although in computer science world living with this idea of like version control um measuring improvement is also a really fantastic way and that, if you can articulate your your students improvements goals, it actually lets you individualize the learning.

B

It means that your top students that you want to be able to be like you, have a great job. You can challenge, but you can also say those students that are coming with a weaker background.

B

If they have improved more, maybe they should be getting a better grade. Okay, the other piece uh is- and I find this challenging and I'm sure other faculty do as well establishing your expectations before you give out assignments and before early in the course. So this is this idea of like presenting a rubric when you assign something not when you go into grade it, and what this does is help.

B

Students know how to essentially structure their results to meet what you're expecting, and it makes your grading easier, because it looks like what you want and it makes their work easier because they know what not to focus on so it's sort of a nice feedback loop, but it also requires a lot of pre-prep, which is not something that um I at least am always fantastic about, and I I don't know how other people are I'm seeing cynthia nod, because I can see her picture so at least someone is agreeing that this is.

C

B

For other people um all right, so I I want to make sure that I think a bunch of the literature analysis and some of the statistical stuff came out of an nsf-funded grant uh and collaborators did a lot of this work and a lot of the um actual analysis happened this past summer with a bunch of uh undergrad reu students and things like this.

B

There is no paper out yet successfully. um It got rejected from harvard data science review. So I have a white paper. If you want. You know a draft of some of this stuff with the detailed literature analysis and statistical analysis survey that I'm happy to share individually, um but uh right now, there's not a published citation that I can point you to thinking about that.

B

That first part about the what the what to assess and that's I'm going to stop there, because I think I've talked long enough and I would be happy to answer questions if you.

D

A

As well I mean we can just share it with those on the call or um I've been here today and it's if, if that's something people be interested in too, I think knowing those 15 topics would be helpful. You know some people are going through.

B

Yes, I can probably share that with you. Let me check with my collaborators first because, like I said we don't have it actually, only one journal has seen a pre-copy of it, and so it's not necessarily out for confirming so um and we're in the process of resubmitting it. So we're hoping it'll be out soon, because we know people want it, um but I will either share it directly with you or if individual people know that they want it.

B

For looking at some of the details, I'm happy to share with individuals on a sort of a case-by-case basis.

A

Sounds good, um so we have a little time for questions here and this one we really wanted to get people's um experience or where you are maybe in uh assessing or how you're thinking about assessment program. This day has been all about. You know really looking at courses but also looking at how you might assess these uh courses. So thank you, carl for a lot of that.

A

Also the background just on edison, I was a a part of like interacting with that eu team when they were doing it, so it was actually a two-year um eu project uh grant that they ran and they ran studies on industry across all of the eu member states um and got thousands of responses from practitioners, so they actually did a really detailed framework on what was out what people needed in the actual workforce and translating that into education.

A

So a lot of their you know, eu sponsored schools have started to use that framework as well. So it's been a good um group that does the edison project as well. They have a few other things too. Besides the framework that are up there like a list, there's a lot.

B

Of fantastic stuff up there, it's it's a matter of what you're looking at, and I mean you could do a lot worse than just saying. I'm going to take out the whole eu thing and say this is what I want to plop down from my data science program.

B

The key is that they, I think the thing that I find somewhat struggle is: they did a really great job of designing if you're building from the ground up an entire program, it's a little bit less aligned if you're, trying to take stuff that you already have in existence and massage it into a good data science program.

A

Yeah they're very fundamental fundamentals, up type of thing and way back. They were thinking of you know implementing an api for this. I don't know how far they uh the project ended before they did the api, so someone would have had to volunteer um so but, and still it was a very comprehensive, they did all the things that they were looking for. So I wanted to go around and usually do that at the beginning, but I figured this time.

A

We do it at the end since and see, um and you know introduce, but also if you have questions about assessment or experience with assessment. We are very interested in that, for you know, as people are starting programs or feedback on how some of these things have been done, so I'm.

B

A

Go around because I like calling people's names everywhere you on my my screen. So cynthia do you want to go first.

E

Hi everyone I'm cynthia searcy, I'm associate dean for academic innovation and strategy in policy study school. So we're not. You know: data science in a business school or in arts and sciences and computer science. So we have built recently or building out a curricula for our students, who, I would say, is like data science lite, um but we have used this framework uh for the edison framework for identifying.

E

You know the knowledge and skills that students would need to. You know have a minor in um and policy analytics as we're calling it, and so it's been very helpful and then, of course, hearing about your courses a particularly earlier hunter's course I'd love to get your your syllabus. um You know because we are you know we have. We have economists, we have con matricians, and um so we have people who are coding and doing things. But you know our students aren't necessarily learning python and data science.

E

So um we I mean some of the harder coding skills that we would like our students to be able to learn, and but you know creating all that curricula from scratch all on our own is so intensive uh and so we'd love to be able to share those those curricula. Thank you.

A

And uh so john yeah eagley.

C

Hi so uh carl set up a data science program previously at valparaiso university, where he is no longer so I'm taking the handoff from him and uh trying to keep it going and expand the program.

A

A

Welcome john claudia.

F

Hi everyone, uh claudia schultz, from the university of virginia school of data science.

F

My role is director of research program, so I work mostly with with faculty on their grant proposals, but I'm also work with corporate sponsors on our sourcing projects for our capstones, because our students do you know experiential learning uh with real world projects, I'm happy to answer any questions. We are about to go to our faculty center with a phd program.

F

um We have a minor um and next year we'll be developing a major but most of our students right now at their master's level um and uh happy to connect to cynthia to our policy school, which has a lot of data science courses, they're very quantitative over at baton.

A

Nice, thank you, claudia yeah, we'll have to have you come and speak talk about some of those programs too.

G

um Yes hi, um so we have a data science concentration within our math majors, both at the undergraduate and graduate level. So that's what we have right now, but we are also working on creating a separate data. Science, slash computer science, some combination uh program- and that is in the works.

A

And feel free anyone that hasn't actually questions about assessment. um You can put it on the chat and interrupt my calling people um as well, so that you know, if you have a question for the speakers um or something to share about assessment, feel free to just say: hey. I want have something to say: I'm just going around as we do this so susie.

D

Hi, I'm from the university of tennessee, and I am associate dean of research for our college of communication and information and we're one of four colleges that are in the process of putting together an mouth and we're establishing an under a interdisciplinary undergraduate data science program that is, does not house in any one particular college, but is actually a partnership of four lead colleges and in uh several other colleges that are joining us on this journey. So these have been great sessions. I haven't made them all, but the ones I've made have been awesome.

D

A

D

A

Love the school pride in the background nick, I know you've worked on, you know some things around assessment too, and in the statistics space we talked about computing, but.

H

Yeah again, I'm sorry, I I I was only able to join partway through carl's um talk, but um um I put a note into the um um the the companion document, um the chapters four and five of the national academies, data science for undergraduates.

B

H

About assessment, not to the level of depth that we need- and so I think these conversations are, are really important and, more generally, I think, as we particularly as we go out to community colleges and try to understand that transfer from from associates programs to four-year colleges. It's gonna, be that's gonna, be really important.

H

The other part that I'm wondering about is ways in which some of these conversations can dovetail with other ongoing efforts uh to blow up the stem curriculum more generally, to make sure it's much more inclusive, welcoming um the high impact practices are part of that. um It's it's difficult to be teaching the kind of capstone-like courses, but they're so critical for our students. How do we kind of make sure that's and leverage stuff we have going on from other colleagues outside of data science.

A

Yeah for sure- and you know that's another thing we'll put in the chat- they you know and they see uh envisioning the data, science, discipline and some of the work you know carl or nick was there as uh for keeping data science broad.

A

um They put some of that information into the group, but we actually ran a workshop for looking at inclusive or uh under um minority, serving primarily teaching focused institutions and how to look at the challenges and opportunities for doing data science programs at those institutions, and so that is a separate report that you can actually find on the south hub website or you can find references to it to in the nac.

A

But looking at which competencies we actually broke down, uh which types of school um mentioned, what types of of challenges and which types of opportunities you know just to say that it's not symmetric what the challenges are and what the opportunities are in different places. So you can kind of get that sliding scale of what things may made the list for those I'm.

B

Going to second nick's comments about the the national academy report that those are that's a really great place to think about the like some of the big picture, how leveled at the curriculum level and then pulling out some of those content. The like the topical things from some of the places that I was highlighting.

A

Yeah and so we'll put both of those yeah thanks nick and then we'll put those in the ether pad too. So there's the link to the national academies. um Let's see where people moved around, so venkata is saying your name right.

G

Hello. Everyone, yes, can you know, can you give me okay? um This is venkata and ludi from alabama a m university um yeah, it's nice uh to get insights into. You know the assessment side of it, um even though I've been um part of as a faculty and as well as the chair of the department for some years in the into the assessment of computer science courses, but I think the data science is, you know a recent thing and we just added some courses, but I've been uh for the last 10 years.

G

My interest in big data is mainly uh of the cybersecurity side that we've been doing. We had a concentration in it and my master's students actually do this related research into you know using python and machine learning techniques. For you know this big data in uh you know whether it's malware or email, phishing or website phishing data, so mostly uh cyber security related domain.

G

um But the assessment uh you know, uh insights really help uh it's interesting because it's a new field, probably I think it's gonna help all the universities in general, who are trying to add data science to their curriculum.

A

Yeah, thank you for sure yeah. Thank you. um Monica brown.

F

Hi, I'm I'm monica brown and I teach at st catherine university in st paul minnesota and we are trying to launch an interdisciplinary data science program so just very grateful for all these wonderful resources and and continue to investigate and use them.

A

Yeah, thank you mark.

G

It's good. Let me turn my camera around yes good morning. My name is mark medina, I'm a faculty and a chair of the computer science program at the university of the virgin islands. Yes, so we launched a new program, a new minor in data science in fallout 2020, so we've not started assessment yet so I think we're going to use a lot of you resources, yeah.

A

Yeah, wonderful! Thanks mark uh we went out there for a part of our data up program, probably a year.

G

With dr lombardi, yes again any time.

A

G

C

Hello, how are you uh yeah al heron, uh executive director of semtech foundation also on the board with uh the career institute, we're here in the seattle area, and we are working with a number of small non-profits cbos that are focusing on bipod communities to help bring uh early on uh introductions to and drive interest in computer science, uh in particular when working with the breakfast group, which has a program called project: mister uh for uh high school high school students, uh young black males uh and by working with the seattle colleges and with the university of washington, we're building the opportunity to drive scholarship driven.

C

You know, certifications, and uh this is always helpful for me to get to make this. I I was about 20 minutes late today and I apologize for that. But I'm glad I'm glad you were recording it and I always like to see or not.

A

Yeah no and you probably go back uh hunter. I think you guys are in the same place he's at university of washington who's our first speaker and they're expanding their classes to high schools. I don't know if you're connected already. Maybe you are.

G

I don't think we met yet, but we should definitely talk absolutely yeah.

A

Okay, uh anna ana, am I saying please for not tell me anything.

A

Anna jot, maybe.

A

Okay, if you come back on move on, florence want to introduce, say hello.

A

Hey there all right, so I think we've got the people. Some people are on mute. Maybe they'll come back for the recordings, um but that's all for today. I think we're always love to give people three minutes back in their day whenever I can, but we will start posting again.

A

I think we, you know for the new year, we'll have these recordings up, but really thinking about you know what are some of the common themes we're trying to think through from all of the meetings like looking back at the notes so probably closer to the summer, we'll have a meeting where we go through what the topics, the main topics have been of interest and trying to work through what things.

A

Maybe this group could do to synergize or bring some of these things together, or you know, work and bring bring to bear a lot of this expertise that is so overflowing, and the group always excited about what happens. uh So, thank you all and happy friday and uh I'll talk later.

C

Very good, thank you.

G

Thank you. Thank you.