{"id":2474,"date":"2023-08-04T15:10:01","date_gmt":"2023-08-04T08:10:01","guid":{"rendered":"https:\/\/www.cloudaja.id\/artikel\/?p=2474"},"modified":"2023-08-04T15:10:01","modified_gmt":"2023-08-04T08:10:01","slug":"robots-txt-pengertian-fungsi-dan-cara-kerja","status":"publish","type":"post","link":"https:\/\/www.cloudaja.id\/artikel\/robots-txt-pengertian-fungsi-dan-cara-kerja\/","title":{"rendered":"Robots.txt: Pengertian, Fungsi, dan Cara Kerja"},"content":{"rendered":"\n<p class=\"wp-block-paragraph\">Dalam era teknologi dan internet yang semakin maju, kita sering mendengar tentang istilah &#8220;robots.txt.&#8221; Namun, bagi kebanyakan orang, istilah ini mungkin terdengar asing dan misterius. Apa sebenarnya robots.txt, dan apa fungsi serta cara kerjanya? Artikel ini akan menjelaskan semua hal tersebut.<\/p>\n\n\n\n<div id=\"ez-toc-container\" class=\"ez-toc-v2_0_89 counter-hierarchy ez-toc-counter ez-toc-grey ez-toc-container-direction\">\n<div class=\"ez-toc-title-container\">\n<p class=\"ez-toc-title\" style=\"cursor:inherit\">Daftar Isi<\/p>\n<span class=\"ez-toc-title-toggle\"><a href=\"#\" class=\"ez-toc-pull-right ez-toc-btn ez-toc-btn-xs ez-toc-btn-default ez-toc-toggle\" aria-label=\"Toggle Table of Content\"><span class=\"ez-toc-js-icon-con\"><span class=\"\"><span class=\"eztoc-hide\" style=\"display:none;\">Toggle<\/span><span class=\"ez-toc-icon-toggle-span\"><svg style=\"fill: #999;color:#999\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" class=\"list-377408\" width=\"20px\" height=\"20px\" viewBox=\"0 0 24 24\" fill=\"none\"><path d=\"M6 6H4v2h2V6zm14 0H8v2h12V6zM4 11h2v2H4v-2zm16 0H8v2h12v-2zM4 16h2v2H4v-2zm16 0H8v2h12v-2z\" fill=\"currentColor\"><\/path><\/svg><svg style=\"fill: #999;color:#999\" class=\"arrow-unsorted-368013\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"10px\" height=\"10px\" viewBox=\"0 0 24 24\" version=\"1.2\" baseProfile=\"tiny\"><path d=\"M18.2 9.3l-6.2-6.3-6.2 6.3c-.2.2-.3.4-.3.7s.1.5.3.7c.2.2.4.3.7.3h11c.3 0 .5-.1.7-.3.2-.2.3-.5.3-.7s-.1-.5-.3-.7zM5.8 14.7l6.2 6.3 6.2-6.3c.2-.2.3-.5.3-.7s-.1-.5-.3-.7c-.2-.2-.4-.3-.7-.3h-11c-.3 0-.5.1-.7.3-.2.2-.3.5-.3.7s.1.5.3.7z\"\/><\/svg><\/span><\/span><\/span><\/a><\/span><\/div>\n<nav><ul class='ez-toc-list ez-toc-list-level-1 ' ><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-1\" href=\"https:\/\/www.cloudaja.id\/artikel\/robots-txt-pengertian-fungsi-dan-cara-kerja\/#Pengertian_Robotstxt\" >Pengertian Robots.txt<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-2\" href=\"https:\/\/www.cloudaja.id\/artikel\/robots-txt-pengertian-fungsi-dan-cara-kerja\/#Fungsi_Robotstxt\" >Fungsi Robots.txt<\/a><ul class='ez-toc-list-level-3' ><li class='ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-3\" href=\"https:\/\/www.cloudaja.id\/artikel\/robots-txt-pengertian-fungsi-dan-cara-kerja\/#Pengaturan_Akses\" >Pengaturan Akses<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-4\" href=\"https:\/\/www.cloudaja.id\/artikel\/robots-txt-pengertian-fungsi-dan-cara-kerja\/#Efisiensi_Crawl\" >Efisiensi Crawl<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-5\" href=\"https:\/\/www.cloudaja.id\/artikel\/robots-txt-pengertian-fungsi-dan-cara-kerja\/#Perlindungan_Data\" >Perlindungan Data<\/a><\/li><\/ul><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-6\" href=\"https:\/\/www.cloudaja.id\/artikel\/robots-txt-pengertian-fungsi-dan-cara-kerja\/#Cara_Kerja_Robotstxt\" >Cara Kerja Robots.txt<\/a><ul class='ez-toc-list-level-3' ><li class='ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-7\" href=\"https:\/\/www.cloudaja.id\/artikel\/robots-txt-pengertian-fungsi-dan-cara-kerja\/#Contoh_robotstxt\" >Contoh robots.txt<\/a><\/li><\/ul><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-8\" href=\"https:\/\/www.cloudaja.id\/artikel\/robots-txt-pengertian-fungsi-dan-cara-kerja\/#Kesimpulan\" >Kesimpulan<\/a><\/li><\/ul><\/nav><\/div>\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Pengertian_Robotstxt\"><\/span>Pengertian Robots.txt<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">robots.txt adalah sebuah file teks yang berfungsi sebagai protokol standar untuk mengontrol aktivitas perayapan (<em>crawling<\/em>) robot atau spider yang dilakukan oleh mesin pencari dan bot internet lainnya. File ini ditempatkan di akar (<em>root<\/em>) situs web dan memberi instruksi kepada robot tentang halaman mana yang diizinkan atau dilarang untuk diakses.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Dalam istilah sederhana, robots.txt bertindak sebagai peta jalan bagi bot web. Robot dari mesin pencari seperti Google, Bing, Yahoo, dan lainnya akan mengunjungi halaman ini sebelum memulai proses crawling untuk memahami batasan-batasan situs web tersebut. Jika robots.txt tidak ada di situs web, bot web akan mengasumsikan bahwa semua halaman dapat diakses.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Fungsi_Robotstxt\"><\/span>Fungsi Robots.txt<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-full is-resized\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/www.cloudaja.id\/artikel\/wp-content\/uploads\/2023\/08\/image.png\" alt=\"\" class=\"wp-image-2480\" width=\"538\" height=\"406\" srcset=\"https:\/\/www.cloudaja.id\/artikel\/wp-content\/uploads\/2023\/08\/image.png 900w, https:\/\/www.cloudaja.id\/artikel\/wp-content\/uploads\/2023\/08\/image-300x227.png 300w, https:\/\/www.cloudaja.id\/artikel\/wp-content\/uploads\/2023\/08\/image-768x580.png 768w\" sizes=\"auto, (max-width: 538px) 100vw, 538px\" \/><\/figure>\n<\/div>\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Pengaturan_Akses\"><\/span>Pengaturan Akses<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Fungsi utama dari robots.txt adalah mengatur hak akses bagi robot web. Dengan adanya file ini, pemilik situs dapat membatasi akses robot ke beberapa bagian situs yang mungkin tidak ingin diindeks atau ditampilkan dalam hasil pencarian.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Efisiensi_Crawl\"><\/span>Efisiensi Crawl<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Dengan memberikan instruksi yang jelas pada robots.txt, situs web dapat membantu mengarahkan bot web untuk memfokuskan crawl pada halaman-halaman yang dianggap lebih relevan atau penting. Hal ini dapat membantu menghemat sumber daya dan waktu robot dan memastikan situs Anda diindeks dengan lebih efisien.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Perlindungan_Data\"><\/span>Perlindungan Data<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Beberapa bagian situs web mungkin berisi informasi sensitif atau data pribadi yang tidak boleh diakses oleh bot web. Dengan menggunakan robots.txt, Anda dapat mencegah informasi sensitif ini dari akses yang tidak diinginkan.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Cara_Kerja_Robotstxt\"><\/span>Cara Kerja Robots.txt<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"460\" height=\"284\" src=\"https:\/\/www.cloudaja.id\/artikel\/wp-content\/uploads\/2023\/08\/image-1.png\" alt=\"\" class=\"wp-image-2482\" srcset=\"https:\/\/www.cloudaja.id\/artikel\/wp-content\/uploads\/2023\/08\/image-1.png 460w, https:\/\/www.cloudaja.id\/artikel\/wp-content\/uploads\/2023\/08\/image-1-300x185.png 300w\" sizes=\"auto, (max-width: 460px) 100vw, 460px\" \/><\/figure>\n<\/div>\n\n\n<p class=\"wp-block-paragraph\">Proses kerja robots.txt relatif sederhana. Ketika bot web dari mesin pencari atau entitas lain mengunjungi sebuah situs web, langkah-langkah berikut terjadi:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Mengakses robots.txt: Bot web akan mencari file robots.txt di direktori utama situs web dengan mengunjungi <strong>http:\/\/namadomainanda.com\/robots.txt<\/strong>.<br><br>Setelah file ditemukan, bot web akan membaca isinya untuk memahami instruksi yang diberikan. File ini dapat berisi beberapa perintah, termasuk instruksi untuk akses penuh atau akses terbatas ke halaman-halaman tertentu.<br><br>Bot web akan mengikuti perintah yang tercantum dalam robots.txt. Jika ada halaman yang diizinkan untuk diakses, bot akan memulai proses crawling di halaman tersebut. Sebaliknya, jika ada halaman yang dilarang untuk diakses, bot akan menghindari halaman tersebut dalam proses crawlingnya.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Contoh_robotstxt\"><\/span>Contoh robots.txt<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Berikut adalah contoh sederhana file robots.txt:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>User-agent: *\nDisallow: \/admin\/\nDisallow: \/private\/\n<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">Dalam contoh ini, file robots.txt memberi perintah kepada semua bot web <strong>(User-agent: *)<\/strong> untuk tidak mengakses halaman dengan URL yang dimulai dengan <strong>\/admin\/<\/strong> atau <strong>\/private\/<\/strong>.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Kesimpulan\"><\/span>Kesimpulan<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Robots.txt adalah instruksi teks yang penting dalam dunia internet yang mengontrol aktivitas perayapan oleh bot web. Dengan memanfaatkan robots.txt dengan bijaksana, Anda dapat mengatur hak akses dan efisiensi crawl di situs web Anda, serta melindungi informasi sensitif dari akses yang tidak diinginkan. Semoga penjelasan di atas membantu Anda memahami pentingnya robots.txt dan bagaimana cara kerjanya.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Jika ada kendala atau pertanyaan, Anda dapat dengan mudah menghubungi kami melalui <a href=\"http:\/\/wa.me\/6283856593722\" target=\"_blank\" rel=\"noreferrer noopener\">live chat <\/a>atau <a href=\"https:\/\/www.cloudaja.id\/memberarea\/submitticket.php?step=2&amp;deptid=1\" target=\"_blank\" rel=\"noreferrer noopener\">open ticket<\/a>. Semoga artikel ini bermanfaat!<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Dalam era teknologi dan internet yang semakin maju, kita sering mendengar tentang istilah &#8220;robots.txt.&#8221; Namun, bagi kebanyakan orang, istilah ini mungkin terdengar asing dan misterius. Apa sebenarnya robots.txt, dan apa fungsi serta cara kerjanya? Artikel ini akan menjelaskan semua hal tersebut. Pengertian Robots.txt robots.txt adalah sebuah file teks yang berfungsi sebagai protokol standar untuk mengontrol [&hellip;]<\/p>\n","protected":false},"author":4,"featured_media":2486,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"seo_booster_metabox":"","footnotes":""},"categories":[4],"tags":[56,65,159],"class_list":["post-2474","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-hosting","tag-hosting","tag-linux","tag-robots-txt"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v28.8 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>Robots.txt: Pengertian, Fungsi, dan Cara Kerja - CloudAja Artikel | Cloud Hosting Indonesia<\/title>\n<meta name=\"description\" content=\"Apa sebenarnya robots.txt, dan apa fungsi serta cara kerjanya? Artikel ini akan menjelaskan semua hal tersebut.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.cloudaja.id\/artikel\/robots-txt-pengertian-fungsi-dan-cara-kerja\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Robots.txt: Pengertian, Fungsi, dan Cara Kerja - CloudAja Artikel | Cloud Hosting Indonesia\" \/>\n<meta property=\"og:description\" content=\"Apa sebenarnya robots.txt, dan apa fungsi serta cara kerjanya? Artikel ini akan menjelaskan semua hal tersebut.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.cloudaja.id\/artikel\/robots-txt-pengertian-fungsi-dan-cara-kerja\/\" \/>\n<meta property=\"og:site_name\" content=\"CloudAja Artikel | Cloud Hosting Indonesia\" \/>\n<meta property=\"article:published_time\" content=\"2023-08-04T08:10:01+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.cloudaja.id\/artikel\/wp-content\/uploads\/2023\/08\/Robots.txt-Pengertian-Fungsi-dan-Cara-Kerja.png\" \/>\n\t<meta property=\"og:image:width\" content=\"1004\" \/>\n\t<meta property=\"og:image:height\" content=\"591\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"Muhammad Habib Ulil A\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Muhammad Habib Ulil A\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"3 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/www.cloudaja.id\\\/artikel\\\/robots-txt-pengertian-fungsi-dan-cara-kerja\\\/#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/www.cloudaja.id\\\/artikel\\\/robots-txt-pengertian-fungsi-dan-cara-kerja\\\/\"},\"author\":{\"name\":\"Muhammad Habib Ulil A\",\"@id\":\"https:\\\/\\\/www.cloudaja.id\\\/artikel\\\/#\\\/schema\\\/person\\\/02ec3c7144d9405a48c99f67fe55f461\"},\"headline\":\"Robots.txt: Pengertian, Fungsi, dan Cara Kerja\",\"datePublished\":\"2023-08-04T08:10:01+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/www.cloudaja.id\\\/artikel\\\/robots-txt-pengertian-fungsi-dan-cara-kerja\\\/\"},\"wordCount\":512,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\\\/\\\/www.cloudaja.id\\\/artikel\\\/#organization\"},\"image\":{\"@id\":\"https:\\\/\\\/www.cloudaja.id\\\/artikel\\\/robots-txt-pengertian-fungsi-dan-cara-kerja\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/www.cloudaja.id\\\/artikel\\\/wp-content\\\/uploads\\\/2023\\\/08\\\/Robots.txt-Pengertian-Fungsi-dan-Cara-Kerja.png\",\"keywords\":[\"hosting\",\"linux\",\"Robots.txt\"],\"articleSection\":[\"Web Hosting\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\\\/\\\/www.cloudaja.id\\\/artikel\\\/robots-txt-pengertian-fungsi-dan-cara-kerja\\\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/www.cloudaja.id\\\/artikel\\\/robots-txt-pengertian-fungsi-dan-cara-kerja\\\/\",\"url\":\"https:\\\/\\\/www.cloudaja.id\\\/artikel\\\/robots-txt-pengertian-fungsi-dan-cara-kerja\\\/\",\"name\":\"Robots.txt: Pengertian, Fungsi, dan Cara Kerja - CloudAja Artikel | Cloud Hosting Indonesia\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/www.cloudaja.id\\\/artikel\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/www.cloudaja.id\\\/artikel\\\/robots-txt-pengertian-fungsi-dan-cara-kerja\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/www.cloudaja.id\\\/artikel\\\/robots-txt-pengertian-fungsi-dan-cara-kerja\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/www.cloudaja.id\\\/artikel\\\/wp-content\\\/uploads\\\/2023\\\/08\\\/Robots.txt-Pengertian-Fungsi-dan-Cara-Kerja.png\",\"datePublished\":\"2023-08-04T08:10:01+00:00\",\"description\":\"Apa sebenarnya robots.txt, dan apa fungsi serta cara kerjanya? Artikel ini akan menjelaskan semua hal tersebut.\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/www.cloudaja.id\\\/artikel\\\/robots-txt-pengertian-fungsi-dan-cara-kerja\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/www.cloudaja.id\\\/artikel\\\/robots-txt-pengertian-fungsi-dan-cara-kerja\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/www.cloudaja.id\\\/artikel\\\/robots-txt-pengertian-fungsi-dan-cara-kerja\\\/#primaryimage\",\"url\":\"https:\\\/\\\/www.cloudaja.id\\\/artikel\\\/wp-content\\\/uploads\\\/2023\\\/08\\\/Robots.txt-Pengertian-Fungsi-dan-Cara-Kerja.png\",\"contentUrl\":\"https:\\\/\\\/www.cloudaja.id\\\/artikel\\\/wp-content\\\/uploads\\\/2023\\\/08\\\/Robots.txt-Pengertian-Fungsi-dan-Cara-Kerja.png\",\"width\":1004,\"height\":591},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/www.cloudaja.id\\\/artikel\\\/robots-txt-pengertian-fungsi-dan-cara-kerja\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/www.cloudaja.id\\\/artikel\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Robots.txt: Pengertian, Fungsi, dan Cara Kerja\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/www.cloudaja.id\\\/artikel\\\/#website\",\"url\":\"https:\\\/\\\/www.cloudaja.id\\\/artikel\\\/\",\"name\":\"CloudAja Artikel | Cloud Hosting Indonesia\",\"description\":\"tips dan trik seputar Cloud Hosting Indonesia\",\"publisher\":{\"@id\":\"https:\\\/\\\/www.cloudaja.id\\\/artikel\\\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/www.cloudaja.id\\\/artikel\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\\\/\\\/www.cloudaja.id\\\/artikel\\\/#organization\",\"name\":\"CloudAja Artikel | Cloud Hosting Indonesia\",\"url\":\"https:\\\/\\\/www.cloudaja.id\\\/artikel\\\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/www.cloudaja.id\\\/artikel\\\/#\\\/schema\\\/logo\\\/image\\\/\",\"url\":\"https:\\\/\\\/www.cloudaja.id\\\/artikel\\\/wp-content\\\/uploads\\\/2022\\\/09\\\/logo-cloudaja.resized.png\",\"contentUrl\":\"https:\\\/\\\/www.cloudaja.id\\\/artikel\\\/wp-content\\\/uploads\\\/2022\\\/09\\\/logo-cloudaja.resized.png\",\"width\":120,\"height\":81,\"caption\":\"CloudAja Artikel | Cloud Hosting Indonesia\"},\"image\":{\"@id\":\"https:\\\/\\\/www.cloudaja.id\\\/artikel\\\/#\\\/schema\\\/logo\\\/image\\\/\"}},{\"@type\":\"Person\",\"@id\":\"https:\\\/\\\/www.cloudaja.id\\\/artikel\\\/#\\\/schema\\\/person\\\/02ec3c7144d9405a48c99f67fe55f461\",\"name\":\"Muhammad Habib Ulil A\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/www.cloudaja.id\\\/artikel\\\/wp-content\\\/uploads\\\/2024\\\/07\\\/elon_mask_avatar_icon_263202-96x96.png\",\"url\":\"https:\\\/\\\/www.cloudaja.id\\\/artikel\\\/wp-content\\\/uploads\\\/2024\\\/07\\\/elon_mask_avatar_icon_263202-96x96.png\",\"contentUrl\":\"https:\\\/\\\/www.cloudaja.id\\\/artikel\\\/wp-content\\\/uploads\\\/2024\\\/07\\\/elon_mask_avatar_icon_263202-96x96.png\",\"caption\":\"Muhammad Habib Ulil A\"},\"description\":\"Salah satu penulis CloudAJA, hobby programming, dan fokus juga di sistem administrator dengan pengalaman beberapa tahun di industri teknologi informasi. Keahlian saya mencakup pengembangan dan manajemen situs web, pemrograman dalam berbagai bahasa seperti PHP, C++, dan Python, serta pengelolaan infrastruktur IT.\",\"url\":\"https:\\\/\\\/www.cloudaja.id\\\/artikel\\\/author\\\/habib\\\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Robots.txt: Pengertian, Fungsi, dan Cara Kerja - CloudAja Artikel | Cloud Hosting Indonesia","description":"Apa sebenarnya robots.txt, dan apa fungsi serta cara kerjanya? Artikel ini akan menjelaskan semua hal tersebut.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.cloudaja.id\/artikel\/robots-txt-pengertian-fungsi-dan-cara-kerja\/","og_locale":"en_US","og_type":"article","og_title":"Robots.txt: Pengertian, Fungsi, dan Cara Kerja - CloudAja Artikel | Cloud Hosting Indonesia","og_description":"Apa sebenarnya robots.txt, dan apa fungsi serta cara kerjanya? Artikel ini akan menjelaskan semua hal tersebut.","og_url":"https:\/\/www.cloudaja.id\/artikel\/robots-txt-pengertian-fungsi-dan-cara-kerja\/","og_site_name":"CloudAja Artikel | Cloud Hosting Indonesia","article_published_time":"2023-08-04T08:10:01+00:00","og_image":[{"width":1004,"height":591,"url":"https:\/\/www.cloudaja.id\/artikel\/wp-content\/uploads\/2023\/08\/Robots.txt-Pengertian-Fungsi-dan-Cara-Kerja.png","type":"image\/png"}],"author":"Muhammad Habib Ulil A","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Muhammad Habib Ulil A","Est. reading time":"3 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.cloudaja.id\/artikel\/robots-txt-pengertian-fungsi-dan-cara-kerja\/#article","isPartOf":{"@id":"https:\/\/www.cloudaja.id\/artikel\/robots-txt-pengertian-fungsi-dan-cara-kerja\/"},"author":{"name":"Muhammad Habib Ulil A","@id":"https:\/\/www.cloudaja.id\/artikel\/#\/schema\/person\/02ec3c7144d9405a48c99f67fe55f461"},"headline":"Robots.txt: Pengertian, Fungsi, dan Cara Kerja","datePublished":"2023-08-04T08:10:01+00:00","mainEntityOfPage":{"@id":"https:\/\/www.cloudaja.id\/artikel\/robots-txt-pengertian-fungsi-dan-cara-kerja\/"},"wordCount":512,"commentCount":0,"publisher":{"@id":"https:\/\/www.cloudaja.id\/artikel\/#organization"},"image":{"@id":"https:\/\/www.cloudaja.id\/artikel\/robots-txt-pengertian-fungsi-dan-cara-kerja\/#primaryimage"},"thumbnailUrl":"https:\/\/www.cloudaja.id\/artikel\/wp-content\/uploads\/2023\/08\/Robots.txt-Pengertian-Fungsi-dan-Cara-Kerja.png","keywords":["hosting","linux","Robots.txt"],"articleSection":["Web Hosting"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/www.cloudaja.id\/artikel\/robots-txt-pengertian-fungsi-dan-cara-kerja\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/www.cloudaja.id\/artikel\/robots-txt-pengertian-fungsi-dan-cara-kerja\/","url":"https:\/\/www.cloudaja.id\/artikel\/robots-txt-pengertian-fungsi-dan-cara-kerja\/","name":"Robots.txt: Pengertian, Fungsi, dan Cara Kerja - CloudAja Artikel | Cloud Hosting Indonesia","isPartOf":{"@id":"https:\/\/www.cloudaja.id\/artikel\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.cloudaja.id\/artikel\/robots-txt-pengertian-fungsi-dan-cara-kerja\/#primaryimage"},"image":{"@id":"https:\/\/www.cloudaja.id\/artikel\/robots-txt-pengertian-fungsi-dan-cara-kerja\/#primaryimage"},"thumbnailUrl":"https:\/\/www.cloudaja.id\/artikel\/wp-content\/uploads\/2023\/08\/Robots.txt-Pengertian-Fungsi-dan-Cara-Kerja.png","datePublished":"2023-08-04T08:10:01+00:00","description":"Apa sebenarnya robots.txt, dan apa fungsi serta cara kerjanya? Artikel ini akan menjelaskan semua hal tersebut.","breadcrumb":{"@id":"https:\/\/www.cloudaja.id\/artikel\/robots-txt-pengertian-fungsi-dan-cara-kerja\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.cloudaja.id\/artikel\/robots-txt-pengertian-fungsi-dan-cara-kerja\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.cloudaja.id\/artikel\/robots-txt-pengertian-fungsi-dan-cara-kerja\/#primaryimage","url":"https:\/\/www.cloudaja.id\/artikel\/wp-content\/uploads\/2023\/08\/Robots.txt-Pengertian-Fungsi-dan-Cara-Kerja.png","contentUrl":"https:\/\/www.cloudaja.id\/artikel\/wp-content\/uploads\/2023\/08\/Robots.txt-Pengertian-Fungsi-dan-Cara-Kerja.png","width":1004,"height":591},{"@type":"BreadcrumbList","@id":"https:\/\/www.cloudaja.id\/artikel\/robots-txt-pengertian-fungsi-dan-cara-kerja\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.cloudaja.id\/artikel\/"},{"@type":"ListItem","position":2,"name":"Robots.txt: Pengertian, Fungsi, dan Cara Kerja"}]},{"@type":"WebSite","@id":"https:\/\/www.cloudaja.id\/artikel\/#website","url":"https:\/\/www.cloudaja.id\/artikel\/","name":"CloudAja Artikel | Cloud Hosting Indonesia","description":"tips dan trik seputar Cloud Hosting Indonesia","publisher":{"@id":"https:\/\/www.cloudaja.id\/artikel\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.cloudaja.id\/artikel\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.cloudaja.id\/artikel\/#organization","name":"CloudAja Artikel | Cloud Hosting Indonesia","url":"https:\/\/www.cloudaja.id\/artikel\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.cloudaja.id\/artikel\/#\/schema\/logo\/image\/","url":"https:\/\/www.cloudaja.id\/artikel\/wp-content\/uploads\/2022\/09\/logo-cloudaja.resized.png","contentUrl":"https:\/\/www.cloudaja.id\/artikel\/wp-content\/uploads\/2022\/09\/logo-cloudaja.resized.png","width":120,"height":81,"caption":"CloudAja Artikel | Cloud Hosting Indonesia"},"image":{"@id":"https:\/\/www.cloudaja.id\/artikel\/#\/schema\/logo\/image\/"}},{"@type":"Person","@id":"https:\/\/www.cloudaja.id\/artikel\/#\/schema\/person\/02ec3c7144d9405a48c99f67fe55f461","name":"Muhammad Habib Ulil A","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.cloudaja.id\/artikel\/wp-content\/uploads\/2024\/07\/elon_mask_avatar_icon_263202-96x96.png","url":"https:\/\/www.cloudaja.id\/artikel\/wp-content\/uploads\/2024\/07\/elon_mask_avatar_icon_263202-96x96.png","contentUrl":"https:\/\/www.cloudaja.id\/artikel\/wp-content\/uploads\/2024\/07\/elon_mask_avatar_icon_263202-96x96.png","caption":"Muhammad Habib Ulil A"},"description":"Salah satu penulis CloudAJA, hobby programming, dan fokus juga di sistem administrator dengan pengalaman beberapa tahun di industri teknologi informasi. Keahlian saya mencakup pengembangan dan manajemen situs web, pemrograman dalam berbagai bahasa seperti PHP, C++, dan Python, serta pengelolaan infrastruktur IT.","url":"https:\/\/www.cloudaja.id\/artikel\/author\/habib\/"}]}},"_links":{"self":[{"href":"https:\/\/www.cloudaja.id\/artikel\/wp-json\/wp\/v2\/posts\/2474","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.cloudaja.id\/artikel\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.cloudaja.id\/artikel\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.cloudaja.id\/artikel\/wp-json\/wp\/v2\/users\/4"}],"replies":[{"embeddable":true,"href":"https:\/\/www.cloudaja.id\/artikel\/wp-json\/wp\/v2\/comments?post=2474"}],"version-history":[{"count":10,"href":"https:\/\/www.cloudaja.id\/artikel\/wp-json\/wp\/v2\/posts\/2474\/revisions"}],"predecessor-version":[{"id":2487,"href":"https:\/\/www.cloudaja.id\/artikel\/wp-json\/wp\/v2\/posts\/2474\/revisions\/2487"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.cloudaja.id\/artikel\/wp-json\/wp\/v2\/media\/2486"}],"wp:attachment":[{"href":"https:\/\/www.cloudaja.id\/artikel\/wp-json\/wp\/v2\/media?parent=2474"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.cloudaja.id\/artikel\/wp-json\/wp\/v2\/categories?post=2474"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.cloudaja.id\/artikel\/wp-json\/wp\/v2\/tags?post=2474"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}