1. Downtime for maintenance is expected for later today. TBD on how long the outage duration will be. Working to minimize, but could last up to several hours.

Server Help Needed!!!

Started by Slyk, Mar 31, 2019, in Issues & Announcements Add to Reading List

  1. Slyk
    Posts: 5,639
    Likes: 18,887
    Joined: Feb 15, 2011
    Location: Detroit -> Chicago.

    Slyk

    God made a prophet.

    Mar 31, 2019
    I worked with @leanaddict a couple months ago to identify that we've been having some pretty severe google crawling issues, which has led to a TON of lost traffic/potential. Looks like this has been going on for a year now... i could really use some help if any of you are familiar with server admin stuff--

    The issue: We're receiving 403 server errors when google tries to crawl the site --
    [​IMG]

    [​IMG]

    [​IMG]

    robots.txt is good, meta tags are good, etc etc. Based on everything I’ve been looking into and reading, it seems like it may be a communication issue on the server, which i know very little about.

    https://aw-snap.info/file-viewer/?p...ioneighty.com/&ref_sel=none&ua_sel=gbot2&fs=0
    ^confirms a 403 crawler error


    Tried working with my server admin @Boolo but he's shit the bed..

    PM me if you can help.

    @BestMembers
     
    #1
    7
    thegoodguy, Alpha, iFlawless and 4 others like this.
    7
    thegoodguy, Alpha, iFlawless and 4 others like this.
    Aug 19, 2019
  2. dna hits
    Posts: 3,916
    Likes: 2,497
    Joined: May 3, 2017

    Mar 31, 2019
    Those errors are shady. You sure this site has the programming right?
    403 is a forbidden error - have you checked your robots.txt file to ensure you have not blocked crawler access?

    You could also try fetching the page as Googlebot to see what response you get.
     
    #2
    1
    Ordinary Joel likes this.
    1
    Ordinary Joel likes this.
    Aug 19, 2019
  3. Lil Squeed
    Posts: 21,490
    Likes: 48,158
    Joined: May 5, 2015

    Lil Squeed

    what it do BABYYYY

    Mar 31, 2019
    Have you tried turning it off and back on again
     
    #3
    29
    Cudder, Carnifex, Eren and 26 others like this.
    29
    Cudder, Carnifex, Eren and 26 others like this.
    Aug 19, 2019
  4. Slyk
    Posts: 5,639
    Likes: 18,887
    Joined: Feb 15, 2011
    Location: Detroit -> Chicago.

    Slyk

    God made a prophet.

    Mar 31, 2019
    upload_2019-3-31_15-50-51.png

    upload_2019-3-31_15-51-24.png

    upload_2019-3-31_15-53-6.png

    ...leads to a link that says the old "fetch as google" tool is no longer available and has been replaced by the "URL Inspection" tool, but that tool is absolute shit.

    if you search google for "sectioneighty", then our root domain doesn't even show in the results.

    robots.txt:
    Code:
    Sitemap: https://forum.sectioneighty.com/sitemap.php
    
    
    User-agent: *
    Disallow: /account/
    Disallow: /members/
    Disallow: /conversations/
    Disallow: /online/
    Disallow: /account/
    Disallow: /attachments/
    Disallow: /goto/
    Disallow: /login/
    Disallow: /best-posts/
    Disallow: /profile-posts/
    Disallow: /posts/
    Disallow: /misc/style?*
    Disallow: /misc/quick-navigation-menu?*
    Disallow: /lost-password/
    Disallow: /help/
    Disallow: /poll/
    Disallow: /likes/
    Disallow: /sportsbook/
    Disallow: /search/
    Disallow: /private.f175/
    Disallow: /admin.php
    Disallow: /search/
    Disallow: /express/
    Disallow: /off-topic.f12/
    Allow: /
    
     
    #4
    1
    Ordinary Joel likes this.
    1
    Ordinary Joel likes this.
    Aug 19, 2019
  5. Slyk
    Posts: 5,639
    Likes: 18,887
    Joined: Feb 15, 2011
    Location: Detroit -> Chicago.

    Slyk

    God made a prophet.

    Mar 31, 2019
    i really don't know much on the server side at all, but i know that boolo at least claimed to have done that a couple times, along with a few other small things but came up empty.
     
    #5
    3
    Ordinary Joel, JB21 and Lil Squeed like this.
    3
    Ordinary Joel, JB21 and Lil Squeed like this.
    Aug 19, 2019
  6. Slyk
    Posts: 5,639
    Likes: 18,887
    Joined: Feb 15, 2011
    Location: Detroit -> Chicago.

    Slyk

    God made a prophet.

    Mar 31, 2019
    robots tester comes up clean:
    upload_2019-3-31_15-59-26.png
     
    #6
    2
    Ordinary Joel and JB21 like this.
    2
    Ordinary Joel and JB21 like this.
    Aug 19, 2019
  7. WheresTheMeat
    Posts: 1,007
    Likes: 2,701
    Joined: Feb 13, 2019

    WheresTheMeat

    12/11/20 Shmurda Season

    Mar 31, 2019
    Strange that goglebot isn't working either.

    You might need to go on another forum for this one, usually that works most of the time.
     
    #7
    1
    Ordinary Joel likes this.
    1
    Ordinary Joel likes this.
    Aug 19, 2019
  8. Slyk
    Posts: 5,639
    Likes: 18,887
    Joined: Feb 15, 2011
    Location: Detroit -> Chicago.

    Slyk

    God made a prophet.

    Mar 31, 2019
    any idea on where's best to go for help? google specific forums haven't been great, aside from a couple people confirming it's a 403 server error, but unclear as to why
     
    #8
    1
    Ordinary Joel likes this.
    1
    Ordinary Joel likes this.
    Aug 19, 2019
  9. dna hits
    Posts: 3,916
    Likes: 2,497
    Joined: May 3, 2017

    Mar 31, 2019
    Yeah that’s odd. I don’t know enough these days to go in the system and mess with it and make it worse tbh. I can do some more into depth research this evening for you at least.

    You don’t think some other forum or forums purposely bugged sxn80 so it doesn’t show up on google? Ik it’s a reach, but there’s past events where forums/subs went to that extent to try and gain more attraction than their competitors.
     
    #9
    2
    Ordinary Joel and Slyk like this.
    2
    Ordinary Joel and Slyk like this.
    Aug 19, 2019
  10. WheresTheMeat
    Posts: 1,007
    Likes: 2,701
    Joined: Feb 13, 2019

    WheresTheMeat

    12/11/20 Shmurda Season

    Mar 31, 2019
    Stack exchange might help. Got some good answers from there a few years ago. Gotta keep looking.
     
    #10
    2
    Ordinary Joel and Slyk like this.
    2
    Ordinary Joel and Slyk like this.
    Aug 19, 2019
  11. Dev One
    Posts: 227
    Likes: 490
    Joined: Mar 30, 2015

    Apr 1, 2019
    I think there's a redirect on https://forum.sectioneighty.com/robots.txt

    If that's where it's located? Robots.txt can't be in a sub folder but they can be on a sub domain.
     
    #11
    1
    Ordinary Joel likes this.
    1
    Ordinary Joel likes this.
    Aug 19, 2019
  12. Dev One
    Posts: 227
    Likes: 490
    Joined: Mar 30, 2015

    Apr 1, 2019
    Nvm - it didn't redirect the second time I tried to access that url

    Another suggestion since my first one was useless: rename robots.txt to robots1.txt to troubleshoot. That will make crawlers ignore robots.txt completely. If it's still broken, you know it's not robots.txt.
     
    Last edited: Apr 1, 2019
    Aug 19, 2019
  13. joeyp363
    Posts: 7,208
    Likes: 8,048
    Joined: Feb 15, 2011

    Apr 1, 2019
    Just hack into the matrix bruh
     
    #13
    3
    dester23, Rick and Ordinary Joel like this.
    3
    dester23, Rick and Ordinary Joel like this.
    Aug 19, 2019
  14. Tyler F u c k i n g Waves
    Posts: 169
    Likes: 371
    Joined: Dec 22, 2018

    Tyler F u c k i n g Waves

    Discord Refugee

    Apr 5, 2019
    Oh so this is why the website's dying

    :eminem2:
     
    #14
    3
    Tonshor, Evad and Ordinary Joel like this.
    3
    Tonshor, Evad and Ordinary Joel like this.
    Aug 19, 2019
  15. Wisheezy
    Posts: 212
    Likes: 270
    Joined: Feb 2, 2017

    Apr 6, 2019
    When did you start noticing this? Surely something has changed for this to happen.

    Can the web admin/server hoster check logs and see if he can see these errors? Maybe they installed or beefed up the firewall at some stage or changed product for this to start? The robots.txt looks OK and I assume that hasn't changed either which makes things point more to your server host.

    403 means: The request was valid, but the server is refusing action. The user might not have the necessary permissions for a resource, or may need an account of some sort.

    @Slyk
     
    Last edited: Apr 6, 2019
    #15
    0 0
    Aug 19, 2019
  16. Slyk
    Posts: 5,639
    Likes: 18,887
    Joined: Feb 15, 2011
    Location: Detroit -> Chicago.

    Slyk

    God made a prophet.

    Apr 9, 2019
    i started noticing it shortly after gaining possession of the site (once koolo stepped down, in october/november-ish)....i didn't have the tools to personally look into it until that point. i did some research into some historical google/ranking data and it led me to believe that it has been happening for a while.

    As of this past month, i no longer have a web admin... boolo/saladin went totally silent on me out of nowhere. he's either dead, in jail, or a piece of shit. he was getting paid and vanished without even completing the time that i already paid him for in advance. it's definitely possible that it's something on that end...again, i just don't know much about that to investigate myself.

    i did change the robots.txt file when looking into this, but i only further opened it -- i'm not sure why some of the additional restrictions were put in place by koolo in the past, so i freed up a handful (i didn't restrict further though).
     
    #16
    2
    Wisheezy and Ordinary Joel like this.
    2
    Wisheezy and Ordinary Joel like this.
    Aug 19, 2019
  17. Slyk
    Posts: 5,639
    Likes: 18,887
    Joined: Feb 15, 2011
    Location: Detroit -> Chicago.

    Slyk

    God made a prophet.

    Apr 9, 2019
    what's also interesting, and likely related, is that the ads on the homepage only show sometimes. at least for me... anybody else notice the same thing?
     
    #17
    1
    Ordinary Joel likes this.
    1
    Ordinary Joel likes this.
    Aug 19, 2019
  18. Koolo
    Posts: 46,599
    Likes: 139,516
    Joined: Feb 14, 2011

    Koolo

    SAS IS THE NEW KANYE

    Apr 9, 2019
    thats normal, you will never have 100% ads coverage, let me look into the webmaster stuff
     
    Aug 19, 2019
  19. Koolo
    Posts: 46,599
    Likes: 139,516
    Joined: Feb 14, 2011

    Koolo

    SAS IS THE NEW KANYE

    Apr 9, 2019
    @Slyk its indexing fine

    upload_2019-4-9_11-32-26.png

    what is the problem exactly? fetching is working fine...if you see tons of fetch issue on a certain old date thats normal..it can happen if the server was down
     
    #19
    1
    Ordinary Joel likes this.
    1
    Ordinary Joel likes this.
    Aug 19, 2019
  20. Koolo
    Posts: 46,599
    Likes: 139,516
    Joined: Feb 14, 2011

    Koolo

    SAS IS THE NEW KANYE

    Apr 9, 2019
    also thats normal, it will show "disallowed" pages as "blocked"
     
    #20
    1
    Ordinary Joel likes this.
    1
    Ordinary Joel likes this.
    Aug 19, 2019
Tags: