Google SEO Office Hours, 25 Mar 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: English Google SEO office-hours from March 25, 2022

Description

This is a recording of the Google SEO office-hours hangout from March 25, 2022. These sessions are open to anything search & website-owner related like crawling, indexing, mobile sites, internationalization, duplicate content, Sitemaps, Search Console, pagination, duplicate content, multi-lingual/multi-regional sites, etc.

Find out more at https://goo.gle/seo-oh-en

Feel free to join us - we welcome folks of all levels!

Timestamps:
00:56 Search Console says I'm ranking #1, but I don't see it.
05:55 Why is my homepage ranking instead of an article?
07:11 Does locating a sitemap in a folder affect indexing?
07:59 I have a new blog, no links. Should I submit in Search Console?
09:22 How fast does the links report update?
10:56 Should the data in Search Console align with Analytics?
16:22 Is there real time data in Search Console?
16:58 Why are disallowed pages getting traffic?
18:12 Where do new soft-404 errors come from?
23:47 Does Google use different algorithms per niche?
26:23 What counts as a click in FAQ rich snippets?
27:40 Best practices for multilingual sites' sitemap files?
29:27 Does link order on a page matter for crawling?
30:31 Why do parameter URLs get indexed?
34:30 Do visually separated links, bold or colored, have more weight?
35:50 Can I reuse sample code in programming tutorials?
37:41 Does out-of-stock affect the ranking of a product page?
39:30 Could a banner on my page affect rankings?
40:38 Is it legal to scrape Google's search results?
41:30 Why did Google pick a different canonical for shared content?
46:33 Are unlinked mentions links too?
49:03 Do links on unindexed pages count?
50:50 What can we do about changed image URLs?
53:34 Would a sitemap of images help with image indexing?
54:25 Does crawl budget affect image re-indexing?

A

All right uh welcome everyone to today's google search central seo office hours hangout. um My name is john mueller. I'm a search advocate on the search relations team here at google in switzerland and part of what we do are these office hour sessions where people can jump in and ask their questions around their website and web search.

A

We have a bunch of things submitted already which we can go through, but also two people with their hands raised already. Let's see first, one I think is cozy. I have your name right.

B

Yes, yes, that's correct, john! Thank you so much all right, um john! I came here like I think it was a week ago and I was talking about uh google not being able to to to show me how many pages are mobile friendly and um by the way I got that fixed and okay. So today my question is about um average position in the google search console.

B

So I a few days ago, I realized that I've got over 60 keyways that are showing that I'm on position one. I still don't understand how that works, because some of the keywords when I'm checking on the on the search results, I don't appear first I've seen some of them are coming from uh the google business profile, but some of them are they're not really appearing. I really want to understand when it says that uh the average position is one. What does is it? What does it really mean?

B

Does it start sometimes say I'm on position one and it takes time to appear or or how does it really work.

A

uh So the average top position is based on what was actually shown to users. So it's not a theoretical ranking where you would appear at number one. If you did it right kind of thing: it's really. We showed it to users and it was ranking at number one in at that time and some sometimes it's tricky to reproduce that which I, I think, kind of makes it a little bit hard in that.

A

Sometimes it's uh for users in a specific location or they search in a specific way, maybe just on mobile or just on desktop, and sometimes it's something perhaps in an image one box or knowledge graph or kind of the local business entry that you mentioned.

A

All of those things are also paces where there could be a link to your website and uh if, if you see something kind of like listed as ranking number one, usually what I would recommend doing is trying to narrow that down into what exactly was searched uh so by using the different filters in search, console figure out which country that was in or what type of query was it uh was it on mobile or on desktop and see if you can try to reproduce it that way.

A

Another thing that I sometimes do is look at the graph over time, so specifically for a query where you're kind of where it says average position ranking number one, but the total impressions or the total clicks don't seem to make much sense and that you think maybe a lot of people are searching but, like you should kind of expect to see a lot of traffic or a lot of impressions. At least that could also be a sign that you were shown at physician number one in whatever way, um but you aren't always shown that way.

A

So that could be that something that was like something very temporary or something that fluctuates a little bit anything along those lines. But it's it's not the case that it's a theoretical position. It's really it was shown in the search results and when it was shown, it was shown like this. Okay.

B

A

B

Its time your ranking or like.

A

B

Okay, like maybe the appearance, uh because one of the things that you mentioned right now that I've seen is that um on some of the queries, I'm ranking number one, I'm not really getting. Maybe any impressions, or some of them are just getting a few impressions and I haven't gotten any clicks, but I'm still it's things still saying that I'm on position, one.

A

Yeah I mean it, it kind of depends on how you were shown there. If, if it's really the case that it's showing you at position number one and you would expect this query- will drive like hundreds of thousands of impressions and you get two or five impressions, uh then to me that sounds almost like some. Some aspect of personalization kicked in maybe a previous query that was used. Maybe the site was previously shown to that specific user, and just for those very few cases, it was actually shown at number one.

B

A

B

Yeah all right, so in this case now am I can I just sit back and can I just sit back and be like okay, I think everything is going well. uh Maybe I should just wait uh a bit more time. Maybe things are going to improve.

A

You could, but my feeling is, it won't just automatically become better just by waiting, because, especially if it's something where it's a little bit fluctuating or sometimes shown sometimes not shown, it means that our algorithms are a bit on the edge and your job in that case would be to convince our algorithms and say well.

A

Actually it's like you should be totally clear that this is a great result to show and uh then just kind of like sitting back you're kind of leaving it up to chance and saying well, we'll see what happens, and maybe your competitor will do better, and then you won't be shown at all. um Maybe at some point your competitor does something really stupid and then you're shown, but then it's kind of out of your control.

A

Whereas if you take the situation where it's kind of fluctuating and say well, I'm going to convince google's algorithms and my side is the best. By far then you're kind of doing things on your side on your terms and uh taking steps to to make it better.

B

Okay, all right so yeah. What I've been doing is I've been adding more content and providing more value and one more one, more thing, john: uh when I'm checking in the search console under pages, the only page, that's actually showing is the home page, mostly for the ones that are ranking. One number one most of most of the crews are ranking on number one. The pages that are showing the page, that's showing mostly is uh is the home page.

B

um I don't understand that because some of the queries, that is, that the homepage seems to be ranking for uh have their own own pages.

A

Yeah, I guess that's just our algorithms kind of seeing your homepage as being the more important page overall, and sometimes that's something where, if you kind of continue working on your website to make it better and better, it's easier for our system to recognize that actually there's a specific page that is more suited for this particular query.

A

But that's kind of like a matter of working on your site and I think working on your site should be something where you're not just generating more and more content, but you're actually creating something better, and that could be by deleting a bunch of content and combining things together. So that's something where it's like. Making improving your website isn't the same as just adding more content all right. Thank you. So much john sure.

C

Phil, hey john um we've got our sitemaps now in the subfolder of our site. I've noticed recently a lot more pages are saying indexed but not submitted.

C

Do you think that might be due to having moved them into the subfolder we used to have them in our route, but due to a technology change, we had to move them.

A

The location of the sitemap file shouldn't really matter, it's something you can put it in a subdirectory. You can put it in a subdomain or a different host. If you want, it depends a bit on how you submit the sitemap file, but if it's, for example, listed in your robot's text file, you can put it anywhere. It doesn't matter if it's in subdirectory or okay.

C

Okay, that helps thanks so much sure.

A

Cool uh barry.

D

Thanks for having me um so I started a new blog, I'm new to blogging, and it's been amazing. Amazing experience to start from scratch. To be honest, um makes me think of new questions anyway, um brand new blog, no links to it. Would you recommend that you submit um urls as you publish them using the google search console, inspect url um then request indexing for new blogs that have no links to it or there's no point: it's not going to really help sitemap wordpress.

A

Yeah, I think you can. I mean it's not that there's any disadvantage of doing that. I I think, if it's a new site, where we really have absolutely no signals or no information about it at all that then at least telling us about the urls is a way of kind of like getting that initial foot in the door. But it's not a guarantee that we'll pick that up.

A

So that's something where probably you know someone else who who is blogging and you can kind of like work together with them and maybe get a link to your site, something along those lines. I that probably, would do a lot more than just going to search, console and saying I I want this url indexed immediately and.

D

I'm trying not to take that or that route I want to try it like brand new. Nobody knows me. I want to see what happens type of thing. Okay. The next question I have is around google search console report links report. I know nobody talks about it anymore, um I'm just curious again. I wanted to see if anybody was linking to me. It shows no external links shows internal links. uh Obviously I'm looking internally and the blog set up that way.

D

But how long does it take typically a new site that has brand new and has links? How long does it usually take google to pick up links in that report? Is it like pretty quick like the performance report and crawl stats report, or is it more like months or so.

A

um I I would have to ask the the team. My understanding is that a lot of the reports in search console are recalculated every three to four days. uh So in terms of like about a week, you should probably see some data there. um The the tricky part, I guess with the links report, is that we show a sample of the links to your site, so that doesn't mean we would immediately populate that one link that we found to your site.

A

uh So that's something where I don't know, I'm kind of curious to see how long it takes to get that picked up. But it's it's not a matter of months and then it's not a matter of kind of like hours like with the performance report. It's more along the line. I'd say like a week, one and a half weeks, something like that should be a reasonable time sounds good. Thank you, cool all right, uh kashish.

E

Hi hi john hi. Thanks for doing this, I actually have two questions and I am going to start my first one by confirming a premise that I have that the google and other websites have stated that the data that we get on google analytics and google search console. um They will not match exactly, but but they will make sense directionally, which means for for your organic search. All your clicks will always be greater than on the sessions that you're getting. Is that understanding, correct.

A

I I guess it depends on what you have set up and.

E

A

You're looking at specifically uh so, if you're looking at it from from a site level, then probably that's about correct. uh If you're looking at it on a per url level on a very large site, then it could happen that those individual urls are not tracked in search console and then you would see slightly different changes or you would see changes over time or some some days. It's tracked some days, it's not tracked in particular, if you have a really large website, but if you look at the website overall, then that should be pretty close.

A

I mean search console measures, essentially what is shown in the search results, the the clicks and impressions from there and.

E

A

Uses javascript to kind of track what what is happening on on the website side and uh those tracking methods are slightly different and they probably have slightly different ways of de-duplicating things. So I would never expect these to line up completely, but overall they they should be pretty close.

E

Quite so, I think I'll give you the context of the problem now. So what I'm guessing I'm running an e-commerce website, um then there's three different kind of pages that I have. One is the product listing pages where you have the list of products and one is the product description page which I'm calling the pdps.

E

So when I'm looking at the organic data, the organic sections that I'm looking that I'm getting so for for the sessions that I get on a product listing page matches directionally with the data that I'm getting on on on the google search console for the plps, but for the pdps product description pages, I'm getting a data difference of around 10x.

E

So, let's see, if I'm getting 500 clicks for a product distance description, page, I'm getting 5 000 organic sessions for those product description pages, so I'm not able to like make sense out of it. What's the probable cause and what's causing this problem, but at the end, like you, said in an overall website level, when I look at the overall organic sessions and our organic clicks, they are they're kind of making sense. But but when I'm looking at the page level, it falls apart for pdps, but for listing pages it is, it is making sense.

A

Now I I think it's it's also a bit tricky with regards to things like canonical urls, and things like that, where I could imagine, if you're looking at individual product pages, that you start to see more and more differences there, that's perhaps another layer that is involved there.

A

The the other thing is sometimes analytics is implemented incorrectly and that can play a role if, if you're, seeing overall kind of on on a website level that it's kind of aligned, then probably it's implemented. Okay, but that might also be something just to to kind of double check. But from from what you're saying I'm assuming it's it's a large e-commerce site- I I don't know yes, but if it's, if it's a really large e-commerce site, then there are definitely limits in search console on how much data we collect per day for sites.

A

So if you drill down on a per url level or on a per query level, then that's something where you might see those differences on a large website.

E

We've got it, but if that were to happen, is that what you're saying is true? I I would have seen the data discrepancy for product listing pages and for for product listing pages as well. But product listing pages, making sense like the clicks and sections are making sense. It's the description pages uh that that's not making sense.

A

Well, I I mean usually, if we have more data for individual pages, then those will be kind of tracked a little bit better in search console. So that's probably, why kind of like the the pages that that are a little bit higher level, essentially probably have kind of more complete data in search console, whereas if you drill down to the the leaf pages on a larger website, then that's probably where you would expect to find more differences.

E

All right, so I I think what what's been juggling for me is for all over the internet that I'm reading I'm seeing that everybody is facing, that they are getting more uh sessions than clicks. But for me it's completely opposite. I'm getting more clicks, uh then sessions.

E

So so I think that's where I'm studying in analytics or how do you? Yes, so yes, the performance sessions in analytics are are more than the clicks that I'm getting on search. Console yeah in a traditional in a traditional problem could be. Your clicks are more than your sessions right.

A

Not necessarily yeah, I okay, I think I think you're you're, probably seeing kind of the effects of a large website and small chunks of the data on there and that's something where you can see kind of these. These variable differences like that.

E

Got it is a second second question: is there a way to check the real time data uh on on google search console no.

A

No in the the performance report is as close as it gets, and I think it goes up to six or nine hours something like that. But it's not real time.

E

You got it and what you're saying is the probable like? The reason for this is that your data is getting limited for the product description, page pages, it's about.

F

That it's the limits of presents.

E

Okay and the next question is um in my robots file, what I've done is. I have disallowed some of the pages certain pages that I've disallowed, but it is quite possible that google had had probably in the in the past, have indexed those pages and whenever I have blogged them like this allowed for and crawling today to this date, I see them like getting organic sessions.

E

Why is that happening and how? How can it fix that? I will do this, something called no index directive, but is that the right way to go about it or how should I? I should appreciate this.

A

If, if these are pages that you don't want to have indexed, then using no index would be better than using the disallow in robots text. uh The no index would be a meta tag on the page, though, so it's a robot's meta tag with no index, and you would need to allow crawling for that to happen.

E

You got it, but so I have to allow that in the robots.txt, but I would have to add the no index metadata on these urls. Exactly got it cool thanks I'll look into it and I think probably we'll see you in the next okay.

A

Thank you thanks uh all right, hazel.

G

Hi john good afternoon, so I've got one question about some software for for you uh errors. So in our jsc bank and there are some sudden increasing errors of software for urls in the coverage report and we used some. We took some sample urls to test in in the mobile friendly test tool and the page is is mobile friendly.

G

However, the version is the the title and description of the tested page is showing in this tool seems to be the older version long long time ago, and and since we couldn't use the you use the live url inspection in jse, so we use extension to monitor the you user agent of google to check the source code of the sample pages.

G

So all looks good to us. Even the title is new, and so is the description. So in this case we couldn't find the reason about this increasing soft for force so anyway otherwise, um or help to identify the problem.

A

It it would be useful to to have some examples. Some sometimes the soft 404s are things like products that are out of stock or not available anymore, and those kind of things can also be picked up as self 404s.

A

uh It could also be that the page is empty or that there's some other error, that's in the dom of the page, if, if there's something happening with javascript, for example, that's that's loading, some kind of an error that says no no information found or a product not available, or something like that.

A

All of that could be could be seen as a sign of the soft 404 um and if, if you're, seeing a rise in the number of soft 404s, my guess is that something in the classifiers for the software for were slightly updated and we're picking up a slightly different pattern on your website and that's something where sometimes using the inspection tools helps you to figure out what might be happening there.

A

If you look at the the full content in the rendered html- and sometimes it doesn't make any sense and if it's something where you you're like well, everything on my site is okay, then those would be really useful examples to have for the team.

G

A

So sorry, um with regards to the the old title being shown, um that would need to be from your side somewhere, because the mobile friendly test and the url inspection test they they don't use any cached data to kind of display. The information that we find there.

A

They really pull that information fresh from your server and it can sometimes happen, depending on your setup, that you have a content delivery network that, for whatever reason, maybe one of the nodes has stale data and uh from that I've seen in the past, that you'd see a difference like that and usually over time, that kind of stale node fixes itself.

A

But that might be something to to double check as well.

G

Okay, good, so you are seeing that when we test a page through this live url, uh the cache page will not be tested right now.

G

It's it's a live, live fetch, yes, okay, so, and the first time we thought it's maybe something about the cache page, but also like I said the older, the title and description is it's a older one, like maybe many many years ago, we dropped that one reasons. Yeah.

A

Yeah, I I think if you see it in the search results that might be from from a cache page, but if you're using the mobile friendly test or the inspect url, it's it's really a live fetch.

G

uh So what what about the the live test? You are uh inspection in jesse. Is it it's also test the live? Url? Yes, good. uh One more question is about this tour of. When will they still be fixed uh any approximate time yeah, because it kind of depends on the on this on this tool to test like some maybe test some life problems, yeah.

A

How do you mean.

G

Yes, because we see some problems that showing something went wrong when we use the tester.

A

G

A

G

A

I don't know I I haven't really heard uh anything recently about that. I assume that was fixed, but I I can double check with the team.

G

A

Good, thank you, cool. Okay. um Let me run through some of the submitted questions and then we'll get back to to some of the people with the hands raised again to see how far we can go. I need to go at the end of the hour, but we'll see how far we can make it.

A

Let's see. Is it true that google has different algorithms for the indexing and ranking of different niches? uh We have two websites of the same type and we've built them with the same process. The only difference is that the two sites are different niches and currently one is ranking, while the other one has lost all ranking.

A

um So I don't think we have anything specific with regards to different niches, but obviously different kinds of content is differently kind of like critical to to our search results. And if you look at something like our quality raters guidelines, uh we we talk about things like your money, your life sites, uh where we we do kind of work to have a little bit more critical algorithms involved in the crawling and indexing and ranking of those sites. But it's not it's not the case that you would say, like a bicycle.

A

Shop has completely different algorithms than I don't know a shoe store, for example, they're they're, essentially both e-commerce type stores. But the thing that you also mentioned in the question is that these are content, aggregator sites and they're built with the same process and some of them work and some of them don't work. That to me feels like it's kind of like I. I don't know your sites.

A

It feels a bit like low effort, affiliate sites where you're just taking content, feeds and publishing them, and that's the kind of thing where our algorithms tend not to be kind of so invested in to make sure that we can crawl and index all of that content, because essentially it's the same content that we've already seen elsewhere.

A

So from that point of view, if you think that might apply to your site, I would recommend focusing on making fewer sites and making them significantly better, so that it's not just aggregating content from other sources, but actually that you're providing something unique and valuable in the sense that if we were to not index your website properly, then the people on the internet would really miss a resource that provides them with value.

A

Whereas if it's really the case that like, if we didn't index your website, then people would just go to one of the other affiliate aggregator sites. Then there's no real reason for us to kind of focus and invest on crawling and indexing the site. So that's something where again, I don't know your websites, uh but that's something that I would look into a little bit more rather than just. Oh, google doesn't like bicycle stores. They like shoe stores instead, um two related questions.

A

uh What counts, as a click for an faq rich snippet, does google ever add links to faq answers, even if there isn't one included in the text.

A

um You link to the the help center article on that which I think is pretty much the definitive source on kind of the clicks and impressions and position counting in search console in general. We count it as a click. If it's really a link to your website and someone clicked and with regards to the rich results, I can't say for sure that we would never add a link to a rich result that we show in the search results. Sometimes I could imagine that we do, but it's not the case that we would say.

A

Oh there's a rich result on this page debra will count it as a click, even though nobody clicked on it. It's really like, if there's a link there and people, click on it and go to your website, that's what we would count and, similarly for impressions, we would count it as an impression if one of those links to your sites were visible in the search results, and it doesn't matter like where it was visible on the page if it's on the top or the bottom of the search results page.

A

If it's theoretically visible to someone on that search, results, page we'll count it as an impression, uh what's the best practice to create sitemap files from multi-language websites right now, we're using subfolder to differentiate the languages and for the sitemap component itself on url value.

A

Should we use the xhtml link, rel alternate hreflang with every available language version of the url, so using subfolders for cycle files is perfectly fine, uh as I mentioned before, um it's it's a reasonable approach to doing it, we don't really care where your sitemap file is located as long as it's kind of in a valid location and a valid location is something either that you're referring to yourself within kind of the setup of your website.

A

So with the robot's text file, for example, you point us at a url somewhere or you're, submitting it in search console and you have the rest of your site also verified in search console. uh Then those sitemap files will be okay.

A

uh With regards to the uh atria flang attributes, you can use those in the sitemap file. You can put them in the html of your page. You can put them in the uh headers of your page, where you do. That is totally up to you. um It's you know some. Some people put it in one place some people in another place.

A

We sometimes also see that it depends a bit on who is actually working on the website in that, maybe you don't have access to the html code, or maybe you can't set the http headers then put it in a sci-fi file is a good way to kind of take advantage of kind of those different locations.

A

I believe for our developer documentation. We use a sitemap file because we can't easily add them to the headers, so we end up putting it there um are there any crawling issues if posts on a category page are sorted by last modification date instead of publish date, um no so in in particular, if we're looking at an html page, the order of the links on there are is not related to the order of our overall crawling of those pages on the website.

A

uh So as long as those links are on the html page and we can follow them, we can essentially pick that up and use that. I I think it might get a little bit tricky if you have things like paginated pages and the newest link is on page 10 of that paginated set. Then we'd first have to crawl all of those 10 pages, leading up to that and then find that new link.

A

That's something I would try to avoid, but if it's all on the same html page and you have kind of like either an order by last modification date or an order by publish date and they're kind of visible on the html page, then that's good enough for us.

A

um Then a question about parameter urls. Why do parameter urls end up in google's index, even though we've excluded them from crawling with the robot's text file and with the parameter settings in search console? um How do we get parameter urls out of the index again without endangering the canonical urls?

A

So I I think there is there's a general assumption here that parameter urls are bad for a website and that's not the case. So it's definitely not the case that you need to kind of like fix the indexed urls of your website, to get rid of all parameter urls. So from from that point of view, it's like I. I would see this as something where you're kind of like polishing the website a little bit to make it a little bit better.

A

But it's not something that I would consider to be critical with regards to the robot's text file and the parameter handling tool. Usually the parameter handling tool is, is the place where you could do these things. My feeling is the parameter. Handling tool is a little bit hard to find and hard to use by people. So personally, I would try to avoid that, and instead use kind of the more scalable approach in the robot's text file, but you're welcome to use it in search console with the robot's text file.

A

You essentially prevent crawling of those urls. You don't prevent indexing of those urls, and that means that if you do something like a site query for those specific urls, it's very likely that you'll still find those urls in the index, even without the content itself being indexed.

A

And I took a look at the the forum thread that you started there, which, which is great, um but uh there you also do kind of this fancy site. Query where you pull out these specific parameter, urls and that's something where. If, if you're looking at uh urls that you're blocking by robot's text, then I I feel that is a little bit misleading because you can find them.

A

If you look for them, but it doesn't mean that they cause any problems or- and it doesn't mean that there's any kind of issue that a normal user would see in the search results so just to elaborate a little bit if there's some kind of term on those pages that you want to be found for- and you have one version of those pages that is indexable and crawlable and another version of the page. That is not crawlable where we just have that url indexed by itself.

A

If someone searches for that term, then we would pretty much always show that page that we actually have crawled into next and the page that we theoretically also have index, because it has it's blocked by robot's text and theoretically, it could also have that term in there. That's something where it wouldn't really make sense to show that in the search results, because we don't have as much confirmation that it matches that specific query.

A

uh So from from that point of view for normal queries, people are not going to see those robotic urls and it's more. If someone searches for that exact url or does a specific site query for those parameters, then they could see those pages.

A

If, if it's a problem that these pages are findable in the search results, then I would use the url removal tool for that. If you can- or you would need to allow crawling and then use a no index directive, robot's text directive to tell us that you don't want these pages indexed but again for for the most part, I wouldn't see that as a problem. It's not something where you need to fix that with regards to indexing.

A

It's not that we have a cap on the number of pages that we index for our website. It's essentially, we we've seen a link to these. We don't know what is there, but we've indexed that url should someone search specifically for that url?

A

uh Is there any direct ranking advantage for improved accessibility, um for example like performance as a url that is more accessible, naturally providing better ux and engagement specifically do links that have features like increased font, weight, underlining or color contrast provide any search advantage over links that are not visually separated from the surrounding text?

A

No, not really so I think accessibility is something that is is important for a website because like if, if you drive your users away with a website that they can't use and they're not going to recommend it to other people, uh but it's not something that we would pick up and use as a direct ranking factor uh when it comes to search, um and maybe that will change over time.

A

That's something where I could imagine the the work that we've done around core web vitals and the page experience uh ranking factor where it could be that at one point we can quantify accessibility a little bit more, and maybe at that point we can use that uh when it comes to ranking, but that's something where at least at the moment, we we don't have any specific plans in that direction.

A

um I have a question about duplicate content. I work in a developer staffing company and we publish technical articles weekly about developer, hacks and tips into our blog section. Is it okay to have the same external source code as a reference to two or more articles?

A

For example, if you have five different articles talking about express.js topics but using the same source code, information from a referring source- uh is this still considered duplicate content, even if it's just a source code from another project. um So I I think there are a few assumptions here as well.

A

On the one hand, that duplicate content is a problem that you need to fix, which is usually not the case, and the other is kind of like if there's a part of a page that is considered duplicate content.

A

Is that a bad thing, and no that's not necessarily a bad thing so from from a practical point of view, what would happen here is we would recognize that a part of the page is the same as other pages that we've seen and if someone were to search for a line of text or piece of text specifically to that shared section, then we would try to pick one of those pages to show in the search results, whereas if someone is searching for something that is relevant to the text around that, then we would try to find the best matching page appropriately from there.

A

uh So from from that point of view, if you're sharing the same code snippet, probably people aren't going to be searching for that code, snippet itself, but rather they're searching for the information around that code snippet.

A

I don't know how to do something special in javascript and then they find that page and as an example, they have that code snippet there, but they don't search for that code snippet itself and from that point of view, that's that's perfectly fine. It's okay to have kind of shared elements on a page talking about the same thing.

A

um Let's say my product page is ranking for a transactional keyword. Would it affect its ranking if the product is out of stock? um And uh I guess the other one question is kind of unrelated. Let's see out of stock, um it's possible, uh let's kind of simplify it uh like that. I I think there are multiple things that come into play when it comes to product themselves, in that they can be shown as a normal search result.

A

They can also be shown kind of as an organic shopping result as well, and if something is out of stock, I believe the organic shopping result might not be shown not 100 sure, and when it comes to normal, the normal search results. It can happen that when we see that something is out of stock, we will assume it's more like a soft 404 era, where we will drop that url from the search results as well.

A

So, theoretically it it could affect essentially the visibility in search. If something goes out of stock, it doesn't have to be the case uh in particular, if you have a lot of information about that product anyway, on those pages, then that page can still be quite relevant for people who are searching for a specific product. uh So it's not necessarily that, like something goes out of stock and that page disappears from search.

A

The other thing is also important. To note here is that even if one product goes out of stock, the rest of the site's rankings are not affected by that.

A

Even if we were to drop that one specific product, because we think it's more like a soft 404 page, then people searching for other products on the site. We would still show those normally it's not that there would be any kind of a negative effect that kind of swaps over into the other parts of the site.

A

um The other question is: could my rankings be influenced by a banner popping up on my page and yes, they could be as well. uh There are multiple things that kind of come into play with regards to banners. On the one hand, uh we we have within the page experience report, we have that aspect of intrusive interstitials and if this banner comes across as an intrusive interstitial, then that could negatively affect the site there.

A

The other thing is that often with banners, you have side effects on the cumulative layout shift like how the page renders, when it's loaded uh or with regards to um the I forgot what the metric is when we show a page, uh the lcp, I think, uh also from the core web vital side with regards to that page.

A

So those are different elements that could come into play here. It doesn't mean it has to be that way, but, depending on the type of banner that you're popping up they, it can happen.

A

I'm building a keyword, research tool that analyzes the search results for competition. However, the api I'd like to use is obviously to scrape google.

A

Is it legal to use such an api, or should I band and creating tools like this? um I can't give you legal advice. So from that point of view, I can't say: is it legal or not? I believe this kind of access would be against our terms of service and how that kind of affects. What you do is is more up to you, but I can't really give you legal advice, uh so I don't know not really sure what to say there.

A

um Okay, let's see, we have a bit more time. uh Maybe I'll go back to some of the people who have their hands raised. uh Let's see, akash.

F

Hi john hi hi, uh so I have a question. uh Let's say if there is a particular page which is uh you know like there's a error in google's uh in google search console which is like you know. Sometimes, google might pick something a different page, what a user has defined.

F

So in that regards, how can we provide the correct signals to the google uh when it comes to? You know, selecting the right page at the canonical. uh I think there is an error right duplicate like google uh choose different canonical, that's user defined. So in order to uh make sure because the pages has the correct, canonical tags uh uh and the coverage reports is different than what the url inspection tool is currently showing, but you you urls. It shows the what the user has correctly defined, but for few pages it shows.

F

You know uh that the google has picked something different uh canonical tag. This is definitely related to a job industry where a lot of hr try to put in the same job description for a particular uh job specific queries.

F

So this is not something which is like controlled by me or like controlled by the team, but uh the from the employer said the job title remains the same, but job description is like copy pasted about several other pages. There could be a duplicate content issue as well, but I don't know how to make sure for specifically job industries how to deal with such problems.

A

Yeah, I I think, first of all kind of making sure that these pages are significantly unique is, is something that is almost like a requirement there uh in the, in the sense that if we think these pages are essentially the same, then we're trying to do you a favor by just picking one url to index it.

A

And you probably don't like that that that we're trying to help you in that regard, but that's something where, if we think these are essentially the same pages and we're saying well, it's like you accidentally submitted multiple pages with the same content, and we try to fix that for you. So the best way to avoid that part is really to make sure that these pages are significantly unique.

A

And if that's something that is based on what people are submitting to your website, then maybe there is a way for you to add some kind of a pre-submit check where you say well this. Actually, this description that you gave me is the same one that you gave me yesterday for a different job uh kind of to push back on that level.

A

That might be an approach, uh but essentially, if the, if the pages are for the most part the same, then it can certainly happen that we fold them together, and you also see this when you have uh kind of like an e-commerce site where you're active in multiple countries and the product descriptions and the products themselves are exactly the same. You just have a different phone number on the bottom of the page uh for the individual country address. Then that's also one of those situations where our systems will say well.

A

This page is essentially the same. We don't need to index all of these variations, so I think that's kind of the first thing that you need to do and, in general, with regards to canonicalization, it's important to keep in mind that there are multiple factors that play a role here. The rel canonical is one thing, but we also look at things like internal and external links. To those pages we try to see what is listed in a sitemap file.

A

What is listed otherwise kind of with uh kind of indirect links on a page with things like structured data.

A

We also look for redirects on on the page, if you're kind of like redirecting from one page to another, then probably the destination page is more relevant and sometimes we also look at things like uh which of these urls looks nicer or which of these urls is cleaner, so in particular, if they're.

A

If we have one page with parameters in the other page without parameters and it's exactly the same thing and we have similar signals, then we will probably try to pick the cleaner url and all of those things need to align and well I mean they don't need to, but ideally they would align so that we can really follow that recommendation, because what we often see or sometimes see at least is that people will put a rel canonical on a page, but internally they'll link to the other version of the page, and then our systems are like.

A

Well, you say this is your canonical, but within your website you're telling us something completely different, uh which of these? Is it, and maybe we'll just pick this one today and maybe we will shift over to another one at some point in the future, so with regards to canonicalization?

A

That's that's always really important, really making sure I I guess first of all for for your particular case is making sure that the content is actually unique so that we don't have to do canonicalization. But if we do canonicalization make sure that you're giving us all of the signals in the right direction,.

F

Cool, so that means that I have really have to spoke to a lot of hrs. I a lot of okay, so my second question is related to a backlinking part, especially so let me just uh break it down into two things: one. uh There are a lot of new sites which are publishing about a particular domain or a particular brand name, but they have not linked that brand name to my particular domain, so those are those so those are just mentions.

F

However, those mentions could be in the form of a brand name or it could be in the form of a person's name as well. So there could be a uh like could be like the ceo. The vps of my company could be uh present at some of the new site you know like they might have spoken to some other new channels as well.

F

So those particular things uh does google uh still able to recognize those mentions those name and since they have been associated with the current company or the current uh particular of the brand that they are working with? uh Does that some kind of influencer has to be like get passed to the particular page?

F

Random, just because you have been mentioned on some other uh places, but you are not linked so that particular kind of entity and if, uh if a news site has written multiple uh articles on us on my particular brand or domain name, let's say three to four articles on on a particular topic, sort of which few pages have been indexed. A few are not indexed.

F

So if the page is not indexed and they have been still linked to us, so are they still count uh from the battling perspective from the uh from the signal point review like these have been referred to, you.

A

I I mean first of all, if there's no link to your pages, then it's not really a link there. So it's not something that we would see kind of like. Oh, it's like a half link worth of mention or something like that. It's it's essentially, there's no link! So there's no signal passing like there would be with any normal link there.

A

So that's, I think, kind of the the first thing to keep in mind that that doesn't mean that kind of these these mentions are bad or that you need to avoid them or anything like that. It's it's really just like we. We don't treat these as links and like that's that's about it uh and it's something where maybe users will pick that mention and they will search for your site and go to your site anyway, but it's it's not a link to to your site. uh So from that point of view it's is very different.

A

I wouldn't put them into the same category.

F

How about uh you have been linked from some pages, but those pages are not been indexed, so, okay, people might able to discover on some other platform, some other cms uh they might have, but they might not index uh due to again uh two favorite issues like crawled. Currently not innocent disorders, not index, but uh but those mentions are.

F

The link has been already present on those particular things, so it is still counted uh just because the page is not indexed, and so those links won't be counted or even if the page is not in it. But if there is a link, those things can be counted as well.

A

Usually that wouldn't count because for a link uh in our systems at least we always need a a source and a destination, and both of those sides need to be canonical indexed urls and if, if we don't have any source at all in our systems, then essentially that link disappears because we don't know what to do with it.

A

So that means if, if the source page is completely dropped out of our search results, then we don't really have any link that we can use there. Obviously, of course, if another page were to copy that original source and also show that link, and then we go off and index that other page, then that would be like a link from that other page to your site, but that original link, if that original page, is no longer indexed, then that would not count as a normal link.

F

I have a follow-up on this. Can I or.

A

Is it still okay or maybe I'll, take matthew.

F

Because I'm running out of time.

A

And I am next next time I should have a bit more time to go through more questions. Sorry.

H

um My question is on the harvesting of images for the recipe gallery, because we've finally identified something which I think has affected some other bloggers and it's really badly affected us, which is that if you have lots of lots of recipes indexed in the recipe gallery and you change the format of your images as the metadata is refreshed, you might have 50 000 of the recipes get new metadata.

H

But there is a deferral process for actually getting the new images and it could be months before those new images being picked up and while they're being harvested, you don't see anything. But when you do a test on google search console it does it in real time and says yeah everything's right, because the image is there. So there's no warning about that. But what it means is you better not make any changes or tweaks to slightly improve the formatting of your image? Url? Because if you do you disappear.

A

um Probably what what is happening there is the the general crawling and indexing of images, which is a lot slower than normal web pages. And if you remove in one image url and you add a new one on a page, then it does take a lot of time to be picked up again and that's that's probably what you're seeing there, what we would recommend in a case like that is to redirect your old urls to the new ones, also for the images.

A

So, if you do something like you have an image url, which has the file size attached to the url, for example, then that url should redirect to a new one, and in that case it's like, we can keep the old one in our systems and we just follow the redirect to the new one.

H

To do that, I guess we have to still send the old one in the metadata, because if we send fresh metadata, I believe you delete the olds on, because our old image urls still work. It's just because we've shipped you the new metadata that you're no longer going there. So I think we have to start sending you the old urls as well.

A

That might be that might be easier to process yeah.

H

Yeah, okay, but it's just yeah it's what the key thing is: there's nothing on the thing work with the previewing and things which says by the way this won't happen in the real world, or I mean it's sort of vaguely gives that caveat. But the fact that images can take so long to be picked up, and I guess you don't see it on the um with google with the web page.

H

These pages still show an image of the web page because you still know how this image goes with this web page, but it's only the recipe metadata where it's the other way around, where it's looking for a particular image, because otherwise it just says we have an image which came from this webpage. So a regular google search result will just show any old image and those came from that web page.

A

Yeah yeah, I mean that's, that's kind of the the way that image search in general works in general yeah, but maybe we should have some kind of a better warning to to catch those kind of issues yeah. If.

H

We had a site map for specifically for images. Will that be of any help in sort of speeding up the harvesting of images.

A

um I don't think so I mean I think it could help us to understand which ones we we need to pick up, but we have to recrawl the web page anyway, and the image would only drop out if we have recalled the web page with a new image url attached to it. So it's I don't think it would speed that up it would it would be more useful if, like if you're documenting the stable state, then that site might file with the image urls would help yeah.

A

But uh if you're you're making a change by changing all the image urls, I I don't think the segment file will change anything there and.

H

The kind of crawl budget essentially there's some algorithms going on deciding how many robots to send where and you just have to live with. However, many it sends you.

A

um Yeah, I I mean what you can do is make sure that your site is really fast in that regard and that's something in the crawl stats report.

A

You should be able to see some of that uh where you see the average response time and I've seen sites that have around 50, milliseconds and other sites that have like 600, 700, milliseconds and obviously, if it's faster, it's easier for us to request a lot of urls because otherwise, like we just get bogged down, because we send- I don't know- 50 google bots to your site at one time and we're kind of waiting for responses before we can move forward all right. So improving performance is yeah, probably helpful. Yeah thanks cool okay.

A

um I I need to jump. I I realize there's still people with your hands raised here uh so sorry, but uh maybe we can catch up uh in one of the next hangouts, uh where I have a bit more time as well. uh It's been great having you all here, uh thanks for all of the questions that you submitted and the the feedback like with the images here uh super useful. um I wish you all a great weekend and hopefully see you all again in one of the future episodes bye. Everyone thanks, john. Thank you.

A