Jan 25, 2017 =================================== WARNING : Please note that I will never recommend using this script to gather content from websites which post news based articles, or in general a lot of text. Google will detect this in almost a split second and your website will most likely be banned before it has the chance too even flourish. [ If you don't see any code displayed in the OP please note that the OP will continue being updated as time progresses, I'm only one f---ing person. ] With the above being noted we can begin with the steps required for you too start, I would like to say that I'm not a f---ing English major so during some parts of this write up my grammar may be a little off or my spelling / word usage may be a little off how ever I recommend you buck the f--- up because nobody else is teaching this s--- in English at all nerds. =================================== Now you're probably wondering how you go about making a website, and now days it's pretty easy you head over too your favorite domain registrar purchase a domain and a server and basically you're almost half way done, but don't worry because I've probably lost you already I'll explain how to do both of those things and realistically starting out.. your budget should be around (200$ usd a year) provided you follow my steps properly making this shouldn't be an issue at all, and you will probably be making it daily at some point if you follow the methods I teach you correctly. =================================== ======= PURCHASING A DOMAIN!! ====== =================================== Now above I mentioned "Domain Registrar" and you're probably lost at what this is, and don't worry I think a lot of people are when they first hear the it, essentially a Domain Registrar is who you purchase your domain from, because they don't just magically appear on the internet you actually have too do some magic behind the scenes steps in order for them to be accessible, in a spoiler tab below I'll put the actual google meaning for "Domain Registrar". Spoiler: Nerd A domain name registrar is an organization or commercial entity that manages the reservation of Internet domain names. A domain name registrar must be accredited by a generic top-level domain (gTLD) registry and/or a country code top-level domain(ccTLD) registry. Now you will probably hear a lot of these being promoted either on YouTube advertisements or for the rare few maybe even on the television how ever I never would recommend you use Godaddy which is the most commonly recommended option via advertisements, I would recommend you go with internet.bs or if you want simple check out options but a bit less security (provided you plan on being on the not so legal side of the stick) you can go with the dudes over at namecheap (still an incredible service just a bit less security when it comes to your information being released 2 who ever request it). Typically a domain will run you anywhere between $8.99*USD / $15.99*USD a year and believe it or not that's not the only payment you need too make in order for your website too go live. =================================== ====== SETTING UP MY SERVER!! ====== =================================== So lets skip past all that mumbo jumbo about you purchasing a domain because once you navigate too the links it's very self explanatory on both websites, I shouldn't have too walk you through each step as if you're a baby. Let's assume you've now purchased a domain how ever you don't have a server too put content on what are you suppose too do now? You can with most domain registrars purchase a hosting account with your domain for an extra fee or you can out source and use a more reliable platform for hosting your content, I recommend someone like OVH hosting provided you're over 18 years of age acquiring an account with them should be extremely hassle free and their costs are really good for all dedicated, vps and shared servers. However if you're not over the age of 18 or you don't really feel like scanning through all of the server options that ovh offers by default you can check out some off shore options like my good friends over at Hosted00.net cheapest solution and insanely good security protecting your information, and server. Let's assume you purchased a server from either of the options above, and decided you wanted too run cpanel on your server, this gives you the ability too check out all of the behind the scene things that your website will come packed with, your MYSQL information, your FTP information and many more options. Upon creating an account with both networks and purchasing a server you will instantly get the information for that server emailed too you, login to the admin panels of those networks and look in your cpanel for a method 2 create a database ( you might see something like Database User + Database name ) fill both of those out, if you have issues just look up on google how 2 create a FTP + PHPMYADMIN account using cpanel, and i'm sure someone will have made a video or an article explaining exactly what you need too do. (Not really interested in covering all of the extremely easy s--- so I'll let you figure some of it out on your own.) =================================== ====== SETTING UP WORD PRESS ====== =================================== Now let's assume you've created a database, you've created an FTP account and figured out how too navigate each of them, you need too now install WordPress this is extremely easy, you head over too the WordPress.org website click the option on the right hand side which says "download word press version ---" then extract the contents from that zip into the ROOT directory on your server (more information you can google how ever it's very self explanatory) if you ask me in a private message or something I might be nice enough too help you out. Provided you've placed all of this information into your root directory you're now ready 2 put your name servers on your domain, with name cheap I'll walk you through the process below which will help you out in finding how 2 do this. Your name servers should come when you purchase a server from each of the server companies provided above, provided you can't find them one again hit my private messages after you've actually made an effort in looking for them and I'll help you out. There should be 2 links which are your name servers in the email NS1.DOMAIN.COM NS1.DOMAIN.COM Copy those one at a time and navigate too this portion of your namecheap acocunt. https://ap.www.namecheap.com/Domains/DomainList after you've reached this page, try and find your domain once you find it click manage on the far right hand side, once you're on manage s--- will start too look a bit clustered don't panic my dude this is normal. Using CTRL+F on your keyboard type in namesevers or just not be r-----ed and view it right in front of your face, on the field for name servers enter the name servers you got in the email from your server hosting company. After you've put them in, hit save wait a couple moments then try viewing your website, if your website displays a WordPress config screen you've done everything correctly so far and deserve a nice long pat on the back. The rest of the steps are very self explained WordPress will walk you through the steps you need, on the final page you will need 2 remember the database information you created in one of the earlier steps and honestly after you've gone through the WordPress config process the hard part of all of this is actually complete, at this point when you view your website again you should see a actual view able website, if you don't see this you've gone wrong somewhere and you can PM me for help or use google which I always recommend, learning to solve a problem on your own is far more important then just begging for help.
Jan 25, 2017 Alright so above, you've created a domain, you've got a server going, and now you just have 2 think of a way 2 either manually post content or get content generated for you automatically. Obviously the second option sounds better then the first one as who wants 2 do anything manually. ( please read the notice in the OP ) In order too gather content from a website we're going to be using a sandbox called simple_html_dom.php for those unfamiliar with PHP or HTML or CSS or any other language, don't worry you don't really need to know too much in order to debug your scraping process, however for this example I will not be scraping any actual websites only a dummy page as I don't want to just give you guys every thing, I do want you to learn also. Let's first start by declaring the file path of the content simple_html_dom provides us, you can do that by adding the following into a file called "scraper.php" ( toss this file anywhere in your root directory and then create a folder called "includes" and throw the simple_html_dom guts into that folder ). Within the scraper.php file add the following PHP: <?phpini_set('post_max_size','512M');ini_set('upload_max_filesize','512M');ini_set('max_execution_time','512M');ini_set('max_input_time','512M');ini_set('memory_limit','512M');set_time_limit(0);include('/home/[your-root]/public_html/[your-site]/wp-config.php');include('/home/[your-root]/public_html/[your-site]/includes/simple_html_dom.php'); Once you've done the above you can start working on a way 2 gather the actual content, below is a very messy and pretty much completely unorganized way of achieving this output, the page we will be gathering content from is (https://370rock.co/) I know I said I was going too be using a test website but I got lazy and decided 2 just google one and found this instead. WARNING: Following code is extremely messy because It's really rushed, I may come back at some point and clean it up, not completely sure too be honest for most code I make open sourced I really don't dedicate too much time into it. PHP: $html = file_get_html('https://370rock.co/'); foreach($html->find('#main-content > div > header > h3 > a') as $element) { print '<br><br>'; echo $url = ''.$element->href; $html2 = file_get_html($url); print '<br>'; The above code will return the link attribute for the actual posts, with simple_html_dom once you get a path going for the script it basically can do the rest on it's own. If you've entered the code correctly into your scraper.php file it should appear something like this in your actual browsers when you view that file. Spoiler: list https://370rock.co/french-montana-diddy-cant-feel-face/ https://370rock.co/prodigy-make-america-great/ https://370rock.co/baauer-kodak-black-many-years-remix-ft-pnb-rock/ https://370rock.co/dee-goodz-acg-dynasty-freestyle/ https://370rock.co/mozzy-hold-ft-g-eazy-yg-lex-aura/ https://370rock.co/jeezy-give-ft-chris-brown/ https://370rock.co/ralo-cant-stop-us-ft-gucci-mane/ https://370rock.co/syd-body/ https://370rock.co/rrose-rrome-rob-2017/ https://370rock.co/sahtyre-mommas-sun/ And if you look at the index page for the site we're gathering content from the post in order are displayed as. Spoiler: list I couldn't scroll out far enough, but I think you get the general idea. Now the way we was able 2 get this data was by right clicking on the title of a post if you're on chrome and I think maybe every other browser has the inspect element option and clicking inspect. After you've done that it should show up as the following provided you're doing it on the same website as I did above in the example. Right click on the link attribute (a href) which is affiliated with the title, and navigate 2 the option which says COPY > then COPY SELECTOR, when you do this, it will give you a giant mess of s--- on most websites, keep fiddling with that content until you get the result you're looking for in your php document. Now that you've gotten the link 2 the post, you should try getting some other data, with most actual blogs that post actual content + images you can grab the image pretty easy by using the code below so just throw this into your php file and maybe it will return a link 2 the image. PHP: // this portion gathers the image if it's set in the meta property.$image = $html2->find('meta[property=og:image]',0);$image = $image->content; // you have 2 use content when gathering data from meta properties.print $image; For the site we're testing on I can confirm that it will return the link of the image, you can check this once again by viewing the file page in your browser for your scraper. Now that you've gotten the image, and the url you're probably stumped at what you do next, probably wondering how do I get the title and the important content? TBH!? It's almost exactly the same as the steps above, if you're looking too gather the actual contents you just right click the content you wanna get inspect element and view around for the selectors and tinker with them until you find the exact path you wanna return, for titles it's pretty much the exact same as the url portion only you print it out differently. View the code example below. PHP: print '<br>'; // this is basically just here because the code is so messy, and you need it 2 display in chunks since we're not storing in an array.// this portion grabs the title from the original a href path as in most times your primary link is obtained through the post name.$title = $html2->find('h1, h2, h3, h4, h5, h6, .mh-custom-posts-small-title',0);$title = $title->plaintext; // instead of using content you can use plaintext 2 just return the text (this makes it easier 2 submit 2 a database.)print $title;// provided you have issues getting the title from the same path as the one you got the content link from, you can navigate 2 the actual post page and right click that post name as well, that's actually the option I used in the above example. The above code will make it so that your content page displays something that looks like this. Spoiler: more https://370rock.co/french-montana-diddy-cant-feel-face/ https://370rock.co/wp-content/uploads/2017/01/face.png French Montana & Diddy – Can’t Feel My Face https://370rock.co/prodigy-make-america-great/ https://370rock.co/wp-content/uploads/2017/01/makk.jpg Prodigy – Make America Great Again https://370rock.co/baauer-kodak-black-many-years-remix-ft-pnb-rock/ https://370rock.co/wp-content/uploads/2017/01/kodkk.jpg Baauer – Kodak Black – Too Many Years Remix Ft. PnB Rock https://370rock.co/dee-goodz-acg-dynasty-freestyle/ https://370rock.co/wp-content/uploads/2017/01/dee.png Dee Goodz – ACG Dynasty Freestyle https://370rock.co/mozzy-hold-ft-g-eazy-yg-lex-aura/ https://370rock.co/wp-content/uploads/2017/01/hoid.jpg Mozzy – Hold On Me Ft. G-Eazy, YG & Lex Aura https://370rock.co/jeezy-give-ft-chris-brown/ https://370rock.co/wp-content/uploads/2017/01/jeee.jpg Jeezy – Give It To Me Ft. Chris Brown https://370rock.co/ralo-cant-stop-us-ft-gucci-mane/ https://370rock.co/wp-content/uploads/2017/01/theyy.jpg Ralo – They Can’t Stop Us Ft. Gucci Mane https://370rock.co/syd-body/ https://370rock.co/wp-content/uploads/2017/01/syd.jpg Syd – Body https://370rock.co/rrose-rrome-rob-2017/ https://370rock.co/wp-content/uploads/2017/01/robbb.jpg RRose RRome – How To Rob 2017 https://370rock.co/sahtyre-mommas-sun/ https://370rock.co/wp-content/uploads/2017/01/mom.jpg Sahtyre – Momma’s Sun With the above information you should have learned the process of inspecting element getting the ID's for the content you're wanting 2 gather data from and then returning that data in your actual code, like I said I'm more a fan of you learning then I am just telling you how too do something so take the above information and run with it as it's extremely valuable. PHP: <?php include('/home/[your-root]/public_html/[your-site]/wp-config.php'); include('/home/[your-root]/public_html/[your-site]/includes/simple_html_dom.php'); $html = file_get_html('https://370rock.co'); foreach($html->find('#main-content > div > header > h3 > a') as $element) { print '<br><br>'; // this is basically just here because the code is so messy, and you need it 2 display in chunks since we're not storing in an array. print $url = ''.$element->href; // this portion gathers the link path and tells teh content below 2 search for certain elements on this page using the $html2 string below $html2 = file_get_html($url); print '<br>'; // this portion gathers the image if it's set in the meta property. $image = $html2->find('meta[property=og:image]',0); $image = $image->content; // you have 2 use content when gathering data from meta properties. print $image; print '<br>'; // this portion grabs the title from the original a href path as in most times your primary link is obtained through the post name. $title = $html2->find('h1, h2, h3, h4, h5, h6, .mh-custom-posts-small-title',0); $title = $title->plaintext; // instead of using content you can use plaintext 2 just return the text (this makes it easier 2 submit 2 a database.) print $title; // provided you have issues getting the title from the same path as the one you got the content link from, you can navigate 2 the actual post page and right click that post name as well, that's actually the option I used in the above example. }?> Overall, the above is how your php file should look completed + note that any data which is stored in any form of value can be accessed and used on your own page (I'll update this a bit later with storing this information into a database and more however my hands really hurt from typing now as I'm a casual.) Provided people wanna keep seeing more of this, I'll explain better + teach you how 2 make custom WordPress themes and more just off simple functions and keeping the job easy for yourself.
Jan 25, 2017 Honestly hoping it helps the site in google a little bit, with simple_html_dom nobodies really explained how to use it and have done it in english, so it could be pretty good for google traffic we'll see though. Provided you wanna edit the OP and make it seem more (SHAZAM) you can send me the updated version via a PM this goes for anyone I guess? I just kinda slopped it together. (typing the coding portion now)
Jan 25, 2017 So you came back from a ban right? Well you should actually use your skills on revenge if so.. However if this an inside plan am down
Jan 25, 2017 The ban was my fault, I provoked @Koolo into banning me and even attacked the site in a negative way after he did so. You can go back and find the thread which originally got me banned, it was for the most part an inside joke completely how ever it escalated 2 something in which it never should have. Generally speaking, I don't want revenge for section eighty I've always wanted what's best for the site. I've just never completely agreed with Koolo's way of getting those things done.
Jan 25, 2017 @Gavin Got a new laptop. But for some reason when I download new songs into my iTunes after a few hours it comes up this song can no longer be located & won't play on my iTunes. Any ideas why this is happening & how to fix this problem?
Jan 25, 2017 I have no idea what any of this means but catch these likes because this is probably really helpful for the right people
Jan 25, 2017 This is beyond off topic lol, how ever I might actually know the issue. 1. Are you purchasing the music? If so I can't see this as being an issue at all as it's stored directly in a file from iTunes. 2. If you're downloading song or albums illegally and storing them in a folder that you're other wise moving somewhere else it will make the song invalid for streaming until you re drag those files back into your iTunes. Provided you're not deleted folders or you're purchasing your audio and the issue is still happening I recommend trying 2 manually re download all songs in your iTunes first that do work or migrating your iTunes library completely from your old computer 2 your new one I believe if you google how 2 do that someone can explain it for you. (actually check here)
Jan 25, 2017 Thanks man, I appreciate the support. This is really the first tutorial for something like this that exists other then the standard documentation for simple_html_dom on their website which looks like it was made by a bunch of monkeys tapping sticks together, so the information above could be useful indeed. The OP version is basically just a slummed down version of how 2 get everything rolling, as I stated in both post that are completed so far.
Jan 25, 2017 I have moved all my old songs onto this new computer in the last week. The songs I downloaded are illegal downloads. When they were in my iTunes I couldn't change name of song or add artwork which was strange. Anything I can do to avoid these problems?
Jan 25, 2017 View the link I posted above and see if the staff recommendations there help you, seems like a f--- load of people have similar problems.
Feb 5, 2017 I stopped working on it half way through, and how it is horrible? The information above explains everything you would need 2 do 2 parse a website, I didn't even begin 2 get started with it.
