rah_sitemap

Project page | Packagist | Twitter | GitHub | Support forum | Donate

Rah_sitemap is a sitemap plugin for Textpattern CMS. The plugin lets you easily generate valid Sitemaps.org XML sitemaps for your site, which help Google and other search engines to index your valuable content. Rah_sitemap maps your categories, sections, articles and even custom URLs of your choosing, and what is best, none of it requires diving into code. All configuration is done from a clean graphical user-interface.

Basics

Rah_sitemap generates a sitemap for your Textpattern website, listing all of its section-, article category- and article-pages. The generated sitemap follows the XML based Sitemap protocol format and is targeted to search engines, opposed to your visitors. The sitemap is meant to help search engines to index your site as it grows and gets more and more various nested pages.

The Sitemap can be configured directly from Textpattern’s Preferences panel, making rah_sitemap very easy to setup and use. The plugin itself takes care of the rest.

Accessing the sitemap

The generated sitemap is publicly accessible from the site’s root. The sitemap can be accessed from two URLs, depending on the site’s Permanent Link Mode. If the site is configured to use clean URLs the sitemap can be accessed using a clean path:

http://example.com/sitemap.xml

The query version is available in both the messy and the clean URL modes:

http://example.com/?rah_sitemap=sitemap

In the above URLs example.com would be the site’s URL as specified in Textpattern’s Preferences.

Immediate boost to search engine visiblity?

Not exactly. Providing Sitemap is to help crawling, finding your site’s pages. Pages that are otherwise hard to discover. Sitemaps itself do not boost content’s visibility.

For a simple, almost static website with just few easily discoverable pages, a sitemap may not be necessary at all. If search engines already can get to know about your pages, you do not really need rah_sitemap or Sitemaps in general. Once a page is indexed, that’s where the sitemap’s work ends in increasing discoverability.

Rah_sitemap is particularly useful when your site has pages that are hard to discover due to being loaded using Ajax, there is way too much distinct content that don’t link to each other, or pages are nested deep in a complex page structure. Rah_sitemap can also help your site to start up when it has very few links point to its pages.

It’s common misconception to think that Sitemaps guarantee that pages will be indexed. This is not the case. Sitemap is a map. Whether a place is marked on a map doesn’t mean someone will actually go there, or when. Like any map, the map is used to find and learn, to increase the future knowledge. Update interval also helps to estimate when your site is updated next and when it should be crawled again.

Normally, you will benefit from submitting a sitemap, but its just one piece of the puzzle. In no case will you get penalized from Sitemap or including the wrong content in it.

List of features

• Build valid XML sitemaps for search engines.
• Simple interface and options integrated to Textpattern’s native preferences panel.
• Filter, include and exclude section, category and article URLs from the sitemap.
• No need to create files nor set permissions, instead of just use the plugin and eventually submit the sitemap URL to Google (via Webmaster tools) and other search engines.

Requirements

Rah_sitemap’s minimum requirements:

• Textpattern 4.5.0 or newer.

Installing

Rah_sitemap’s installation follows the standard plugin installation steps.

1. Download the plugin installation code.
2. Copy and paste the installation code into the Install plugin box of your Textpattern Plugin pane.
3. Run the automated setup.
4. After the setup is done, activate the plugin. Done.

Configuration

After the rah_sitemap is installed, you may want configure to fit your site. For instance, you may want to exclude certain inrelevant articles or sections from the sitemap. The plugin’s settings can be configured from Textpattern’s Advanced Preferences panel, organized under Sitemap section. Sections and Categories can be excluded from the sitemap from their respectful editors.

Sending the sitemap to search engines

Once you have a sitemap up and running, you now should inform search engines about it. There are few ways you can do it. There is Webmaster Tools for Google, you could use a robots.txt directive or do search vendor specific pinging.

To use a robots.txt file to inform about your sitemap’s location, you will need access to create a file at the root of the domain named robots.txt. In that file you would add a Sitemap directive containing an absolute URL to your sitemap:

Sitemap: http://www.example.com/?rah_sitemap=sitemap

Where the http://www.example.com/ is your site’s location as defined in Textpattern’s Preferences panel.

If this all about robots.txt seems complicated, or you don’t have full access to the domain’s root contents, then the easiest and the most user friendly method would be to use each search engine’s own interface and tools. For instance, Google offers Webmaster Tools where you can send in and validate sitemaps. Webmaster Tools has a easy to use web based, graphical user-interface.

Preferences

Rah_sitemap comes with number of preferences which all can be found from your Advanced Preferences panel, organized under a Sitemap section. Rah_sitemap allows excluding sections, categories and articles from the XML sitemap. Following options will be present.

Exclude articles based on fields

The field can be used to exclude articles from the sitemap based on any article field and its value. The option takes a comma-separated list of articlefield: value pairs, where the field is the database field and the value is the field’s value that will be excluded. Available fields include Title, AuthorID, Body, Excerpt, Category1, Category2, Section, Keywords, url_title, custom_1 to custom_10 and Image.

Values used in the option support two wildcard characters. An underscore (_) matches exactly one character, and a percent sign (%) matches zero or more characters.

If you wanted to exclude articles posted to sections named as notes and private or by a user mailer, you could use the following in the field:

Section: notes, Section: private, AuthorID: mailer

Additional URLs

Comma-separated list of additional local site URLs added to the sitemap. Note that a Sitemap only allows local URLs, meaning that any URL used, needs to link to the same domain as where the website itself is located. If a URL is relative and doesn’t start with a HTTP or HTTPS protocol, the site’s URL is prepended to the URL.

Include future articles?

If set to Yes, articles with future publishing date are visible in the sitemap. Please note that by default the article tag doesn’t display future articles, unless its time attribute is explicitly set to future or any.

Include published articles?

If set to Yes, published articles are visible in the sitemap. If both this option and Include future articles? are set to No, no articles will be visible in the sitemap.

Include expired articles?

If set to No, expired articles are not visible in the sitemap.

Exclude sticky articles?

If set to Yes, sticky articles are not visible in the sitemap.

Extras and modules

Rah_sitemap supports modules. The plugin’s own modules can be used to add new functionality.

• Links – displays Textpattern’s links in the sitemap.

For developers

Rah_sitemap offers small set of tools for developers. These tools allow other Textpattern plugins to extend rah_sitemap’s functionality by adding new URLs to the sitemap. The plugin is packaged in a class structure that can be extended if needed, and introduces new Textpattern callback events.

Callback

Rah_sitemap introduces a new public-facing callback event to the Textpattern’s event library named rah_sitemap.urlset. The event is fired before a sitemap is printed out. The callback event can be used with the API to add new URLs to the sitemap.

As with other callback events in Textpattern, hooking to rah_sitemap’s event happens using Textpattern’s callback handling functions, mainly register_callback.

register_callback('abc_function', 'rah_sitemap.urlset', 0, $urls);

fuction abc_function($event, $step, $void, $urls)
{
	$urls['http://example.com/foo/bar'] = '2013-03-04 10:06:30';
}

Custom URL functions

If you are supplying a custom URL function for Textpattern, please note that the URLs the function generates need to meet RFC 3986 and RFC 3987. All URLs should also be entity escaped from special syntax characters using Textpattern’s txpspecialchars function. All URLs Textpattern itself generates follow these specifications, and so should your custom URL plugin.

As rah_sitemap integrates well with Textpattern’s core, it uses the same URL functions as Textpattern. If an URL given to the sitemap doesn’t meet those specification, the sitemap will become invalid.

Changelog

Version 1.3.0 – upcoming

• Changed: Integrated preferences to Textpattern’s native preferences panel, and to Section and Category editors.
• Added: Language strings, interface is now translatable using Textpacks.
• Added: Finnish translation.
• Added: French translation by Patrick Lefevre.
• Removed: Number of extra configuration options including zlib-fixes, compression and custom URL formats.
• Improved: Zlib and gzip options are checked and dealt with automatically.
• Improved: Compatibility with Textpattern 4.5.
• Now requires Textpattern 4.5 or newer.

Version 1.2 – 2011/03/09

• Added: adds site URL to relative article permlinks. Basically a fix for gbp_permanent_links.
• Changed: from permlinkurl_id() to permlinkurl(). Greatly reduced the amount of queries generating article permlinks makes.

Version 1.1 – 2010/10/30

• Fixed issues appearing with the installer when MySQL is in strict mode. Thank you for reporting, Gallex.

Version 1.0 – 2010/10/29

• Slightly changed backend’s installer call; only check for installing if there is no preferences available.

Version 0.9 – 2010/08/25

• Fixed: now correctly parses category tags in category URLs. Thank you for reporting, Andreas.

Version 0.8 – 2010/07/27

• Now compression level field’s label now links to the correct field id.
• Now suppresses E_WARNING/E_STRICT notices in live mode caused by Textpattern’s timezone code when some conditions are met (TXP 4.2.0, PHP 5.1.0+, TXP’s Auto-DST feature disabled, TXP in Live mode). Error suppression will be removed when TXP version is released with fully working timezone settings.
• Now generates UNIX timestamps within the SQL query, not with PHP.
• Changed sliding panels’ links (a elements) into spans.

Version 0.7 – 2010/05/30

• Fixed: now deleting custom url leads back to the list view, not to the editing form.
• Removed some leftover inline styles from v0.6.

Version 0.6 – 2010/05/30

• Rewritten the code that generates the sitemap.
• New admin panel look.
• Now custom permlink modes and custom urls are escaped. Users can input unescaped URLs/markup from now on.
• Now custom URL list shows the full formatted URL after auto-fill instead of the user input.
• Now custom URLs that start with www. are completed with http:// protocol.
• Now all urls that do not start with either http, https, www, ftp or ftps protocol are auto-completed with the site’s address.
• Custom url editor got own panel. No longer the form is above the URL list.
• Added ability to manually turn gzib compression off and change the compression level.
• Added setting to set zlib.output_compression off. See here, thank you for reporting superfly.
• Preferences are now trimmed during save.
• Merged rah_sitemap_update() with rah_sitemap_save().
• From now on all new installations have default settings defined that will automatically exclude link, file and image categories from the sitemap. This won’t effect updaters.
• Changed sitemap’s callback register from pre pretext to callback after it (callback is now textpattern). Now $pretext is set before the sitemap and thus more plugins might work within permlink settings and custom urls.
• When using TXP’s clean URLs, requesting /sitemap.xml.gz and /sitemap.xml URLs will return the sitemap, not just the /?rah_sitemap=sitemap. This will of course require existing fully working clean urls.

Version 0.5 – 2010/03/01

• Added customizable timestamp formats.
• Cleaned backend markup.
• Combined individual preference queries.

Version 0.4 – 2009/04/12

• Added support for custom permlink rules: Now you can easily set any kind of permlink rules for articles, section and categories.
• Added option to exclude future articles.
• Added option to exclude past articles.
• Added option to exclude expired articles.
• Moved Custom URL UI to it’s own page.
• Added multi-delete feature to Custom URL UI.
• Improved Custom URL UI.
• Removed default static appending domain from Custom URL input field.
• Changed TXP minimum requirement to version 4.0.7 (and above). Note that the plugin still works with older TXP versions (down to 4.0.5) if the Exclude Expired articles -option is left empty (unset).

Version 0.3.2 – 2008/10/25

• Fixed view url that still (from version 0.2) included installation address before link.

Version 0.3 – 2008/10/24

• Added option to insert URLs that are outside Textpattern install directory.
• Fixed option to exclude categories directly by type: added forgotten link type.

Version 0.2 – 2008/10/22

• Added option to exclude/include sticky articles.
• Added option to exclude categories directly by type.
• Fixed bug: now shows all categories, and not only article-type, in admin panel.
• Fixed bug: removed double install query (didn’t do a thing, just checked table status twice).

Version 0.1.2 – 2008/09/12

• Fixed article listing bug caused by nasty little typo: now only 4 and 5 statuses are listed.

Version 0.1 – 2008/09/07

• First release.