Server Help Needed!!!

Started by Slyk, Mar 31, 2019, in Announcements Add to Reading List

Thread Status:
Not open for further replies.
  1. Slyk
    Posts: 8,493
    Likes: 29,614
    Joined: Feb 15, 2011
    Location: Detroit -> Cali

    Slyk God made a prophet.

    Mar 31, 2019
    I worked with @leanaddict a couple months ago to identify that we've been having some pretty severe google crawling issues, which has led to a TON of lost traffic/potential. Looks like this has been going on for a year now... i could really use some help if any of you are familiar with server admin stuff--

    The issue: We're receiving 403 server errors when google tries to crawl the site --
    [​IMG]

    [​IMG]

    [​IMG]

    robots.txt is good, meta tags are good, etc etc. Based on everything I’ve been looking into and reading, it seems like it may be a communication issue on the server, which i know very little about.

    https://aw-snap.info/file-viewer/?p...ioneighty.com/&ref_sel=none&ua_sel=gbot2&fs=0
    ^confirms a 403 crawler error


    Tried working with my server admin @Boolo but he's s--- the bed..

    PM me if you can help.

    @BestMembers
     
    #1
    8
    soufyan, thegoodguy, Alpha and 5 others like this.
    8
    soufyan, thegoodguy, Alpha and 5 others like this.
    Mar 28, 2024
  2. dna hits
    Posts: 5,791
    Likes: 3,870
    Joined: May 3, 2017

    Mar 31, 2019
    Those errors are shady. You sure this site has the programming right?
    403 is a forbidden error - have you checked your robots.txt file to ensure you have not blocked crawler access?

    You could also try fetching the page as Googlebot to see what response you get.
     
    #2
    1
    Ordinary Joel likes this.
    1
    Ordinary Joel likes this.
    Mar 28, 2024
  3. Lil Squeed
    Posts: 24,190
    Likes: 57,526
    Joined: May 5, 2015

    Lil Squeed French Montana Stan

    Mar 31, 2019
    Have you tried turning it off and back on again
     
    Mar 28, 2024
  4. Slyk
    Posts: 8,493
    Likes: 29,614
    Joined: Feb 15, 2011
    Location: Detroit -> Cali

    Slyk God made a prophet.

    Mar 31, 2019
    upload_2019-3-31_15-50-51.png

    upload_2019-3-31_15-51-24.png

    upload_2019-3-31_15-53-6.png

    ...leads to a link that says the old "fetch as google" tool is no longer available and has been replaced by the "URL Inspection" tool, but that tool is absolute s---.

    if you search google for "sectioneighty", then our root domain doesn't even show in the results.

    robots.txt:
    Code:
    Sitemap: https://forum.sectioneighty.com/sitemap.php
    
    
    User-agent: *
    Disallow: /account/
    Disallow: /members/
    Disallow: /conversations/
    Disallow: /online/
    Disallow: /account/
    Disallow: /attachments/
    Disallow: /goto/
    Disallow: /login/
    Disallow: /best-posts/
    Disallow: /profile-posts/
    Disallow: /posts/
    Disallow: /misc/style?*
    Disallow: /misc/quick-navigation-menu?*
    Disallow: /lost-password/
    Disallow: /help/
    Disallow: /poll/
    Disallow: /likes/
    Disallow: /sportsbook/
    Disallow: /search/
    Disallow: /private.f175/
    Disallow: /admin.php
    Disallow: /search/
    Disallow: /express/
    Disallow: /off-topic.f12/
    Allow: /
    
     
    #4
    1
    Ordinary Joel likes this.
    1
    Ordinary Joel likes this.
    Mar 28, 2024
  5. Slyk
    Posts: 8,493
    Likes: 29,614
    Joined: Feb 15, 2011
    Location: Detroit -> Cali

    Slyk God made a prophet.

    Mar 31, 2019
    i really don't know much on the server side at all, but i know that boolo at least claimed to have done that a couple times, along with a few other small things but came up empty.
     
    #5
    3
    Ordinary Joel, JEB and Lil Squeed like this.
    3
    Ordinary Joel, JEB and Lil Squeed like this.
    Mar 28, 2024
  6. Slyk
    Posts: 8,493
    Likes: 29,614
    Joined: Feb 15, 2011
    Location: Detroit -> Cali

    Slyk God made a prophet.

    Mar 31, 2019
    robots tester comes up clean:
    upload_2019-3-31_15-59-26.png
     
    #6
    2
    Ordinary Joel and JEB like this.
    2
    Ordinary Joel and JEB like this.
    Mar 28, 2024
  7. Enez
    Posts: 1,809
    Likes: 4,746
    Joined: Feb 13, 2019

    Enez Purple Heats Forever

    Mar 31, 2019
    Strange that goglebot isn't working either.

    You might need to go on another forum for this one, usually that works most of the time.
     
    #7
    1
    Ordinary Joel likes this.
    1
    Ordinary Joel likes this.
    Mar 28, 2024
  8. Slyk
    Posts: 8,493
    Likes: 29,614
    Joined: Feb 15, 2011
    Location: Detroit -> Cali

    Slyk God made a prophet.

    Mar 31, 2019
    any idea on where's best to go for help? google specific forums haven't been great, aside from a couple people confirming it's a 403 server error, but unclear as to why
     
    #8
    1
    Ordinary Joel likes this.
    1
    Ordinary Joel likes this.
    Mar 28, 2024
  9. dna hits
    Posts: 5,791
    Likes: 3,870
    Joined: May 3, 2017

    Mar 31, 2019
    Yeah that’s odd. I don’t know enough these days to go in the system and mess with it and make it worse tbh. I can do some more into depth research this evening for you at least.

    You don’t think some other forum or forums purposely bugged sxn80 so it doesn’t show up on google? Ik it’s a reach, but there’s past events where forums/subs went to that extent to try and gain more attraction than their competitors.
     
    #9
    2
    Ordinary Joel and Slyk like this.
    2
    Ordinary Joel and Slyk like this.
    Mar 28, 2024
  10. Enez
    Posts: 1,809
    Likes: 4,746
    Joined: Feb 13, 2019

    Enez Purple Heats Forever

    Mar 31, 2019
    Stack exchange might help. Got some good answers from there a few years ago. Gotta keep looking.
     
    #10
    2
    Ordinary Joel and Slyk like this.
    2
    Ordinary Joel and Slyk like this.
    Mar 28, 2024
  11. Dev One
    Posts: 227
    Likes: 490
    Joined: Mar 30, 2015

    Apr 1, 2019
    I think there's a redirect on https://forum.sectioneighty.com/robots.txt

    If that's where it's located? Robots.txt can't be in a sub folder but they can be on a sub domain.
     
    #11
    1
    Ordinary Joel likes this.
    1
    Ordinary Joel likes this.
    Mar 28, 2024
  12. Dev One
    Posts: 227
    Likes: 490
    Joined: Mar 30, 2015

    Apr 1, 2019
    Nvm - it didn't redirect the second time I tried to access that url

    Another suggestion since my first one was useless: rename robots.txt to robots1.txt to troubleshoot. That will make crawlers ignore robots.txt completely. If it's still broken, you know it's not robots.txt.
     
    Last edited: Apr 1, 2019
    Mar 28, 2024
  13. joeyp363
    Posts: 15,374
    Likes: 24,259
    Joined: Feb 15, 2011

    Apr 1, 2019
    Just hack into the matrix bruh
     
    #13
    3
    dester23, Ricky and Ordinary Joel like this.
    3
    dester23, Ricky and Ordinary Joel like this.
    Mar 28, 2024
  14. Tyler F u c k i n g Waves
    Posts: 153
    Likes: 328
    Joined: Dec 22, 2018

    Tyler F u c k i n g Waves Discord Refugee

    Apr 5, 2019
    Oh so this is why the website's dying

    :eminem2:
     
    #14
    3
    Wreckless, Evad and Ordinary Joel like this.
    3
    Wreckless, Evad and Ordinary Joel like this.
    Mar 28, 2024
  15. Wisheezy
    Posts: 219
    Likes: 280
    Joined: Feb 2, 2017

    Apr 6, 2019
    When did you start noticing this? Surely something has changed for this to happen.

    Can the web admin/server hoster check logs and see if he can see these errors? Maybe they installed or beefed up the firewall at some stage or changed product for this to start? The robots.txt looks OK and I assume that hasn't changed either which makes things point more to your server host.

    403 means: The request was valid, but the server is refusing action. The user might not have the necessary permissions for a resource, or may need an account of some sort.

    @Slyk
     
    Last edited: Apr 6, 2019
    #15
    0 0
    Mar 28, 2024
  16. Slyk
    Posts: 8,493
    Likes: 29,614
    Joined: Feb 15, 2011
    Location: Detroit -> Cali

    Slyk God made a prophet.

    Apr 9, 2019
    i started noticing it shortly after gaining possession of the site (once koolo stepped down, in october/november-ish)....i didn't have the tools to personally look into it until that point. i did some research into some historical google/ranking data and it led me to believe that it has been happening for a while.

    As of this past month, i no longer have a web admin... boolo/saladin went totally silent on me out of nowhere. he's either dead, in jail, or a piece of s---. he was getting paid and vanished without even completing the time that i already paid him for in advance. it's definitely possible that it's something on that end...again, i just don't know much about that to investigate myself.

    i did change the robots.txt file when looking into this, but i only further opened it -- i'm not sure why some of the additional restrictions were put in place by koolo in the past, so i freed up a handful (i didn't restrict further though).
     
    #16
    2
    Wisheezy and Ordinary Joel like this.
    2
    Wisheezy and Ordinary Joel like this.
    Mar 28, 2024
  17. Slyk
    Posts: 8,493
    Likes: 29,614
    Joined: Feb 15, 2011
    Location: Detroit -> Cali

    Slyk God made a prophet.

    Apr 9, 2019
    what's also interesting, and likely related, is that the ads on the homepage only show sometimes. at least for me... anybody else notice the same thing?
     
    #17
    1
    Ordinary Joel likes this.
    1
    Ordinary Joel likes this.
    Mar 28, 2024
  18. Oldboy
    Posts: 50,757
    Likes: 158,098
    Joined: Feb 14, 2011

    Oldboy Administrator

    Apr 9, 2019
    thats normal, you will never have 100% ads coverage, let me look into the webmaster stuff
     
    Mar 28, 2024
  19. Oldboy
    Posts: 50,757
    Likes: 158,098
    Joined: Feb 14, 2011

    Oldboy Administrator

    Apr 9, 2019
    @Slyk its indexing fine

    upload_2019-4-9_11-32-26.png

    what is the problem exactly? fetching is working fine...if you see tons of fetch issue on a certain old date thats normal..it can happen if the server was down
     
    #19
    1
    Ordinary Joel likes this.
    1
    Ordinary Joel likes this.
    Mar 28, 2024
  20. Oldboy
    Posts: 50,757
    Likes: 158,098
    Joined: Feb 14, 2011

    Oldboy Administrator

    Apr 9, 2019
    also thats normal, it will show "disallowed" pages as "blocked"
     
    #20
    1
    Ordinary Joel likes this.
    1
    Ordinary Joel likes this.
    Mar 28, 2024
Thread Status:
Not open for further replies.