XML Sitemaps for Pligg


Thanks for visiting! If you're new here, you may want to subscribe to my RSS feed. This blog posts regular information about web development, unix/linux, How-tos and patches. Go ahead, subscribe to my feed! You can also receive updates via email, instant messenger, skype or tweeter.

Update: There is be a new version of this module. Click here to get it.

I created a module that generates XML Sitemaps for Pligg ( the well known open source cms used for creating sites similar to digg.com ).

The module generates a sitemap index and sitemaps with all the stories in the database dynamically so nothing is stored on disk and you don't have to set a cron job to generate it.

The sitemaps are updated automatically when a new story is submitted. Because of the structure of the sitemap index and because it contains "lastmod" info, the search engines should only download the latest entries in the index so you shouldn't worry about the module putting too much load on your system.

There is also a "ping" function that will announce google, yahoo and ask.com every time a new story is submitted so that they know they have to download the sitemap. The ping function required a little patching to pligg source code to add some hooks ( only if you use 9.6, 9.7 should already have those hooks ). Here is the diff file in case you use pligg 0.9.6 : pligg submit hooks diff

The module was only tested on pligg 0.9.6, I haven't upgraded to 0.9.7 yet, so if you try this on 0.9.7 let me know how it works, any feedback is appreciated.

Download:

You can download Xml_Sitemaps module from here: xml_sitemaps-0.1.tar.gz and in case you want to verify it here is the md5 sum and the sha256 sum

the code is released under the same license as pligg, so feel free to use it, modify and share.

Installation:

This is pretty straight forward, you have to install this like any other pligg module, just put the .tar.gz file in the modules, un-archive it then go into pligg admin and activate it. If you use pligg 0.9.6 and you want to be able to ping the search engines don't forget to apply the submit hooks patch .

Configuration:

After installation you should be able to access the sitemap index like this : http://yourdomain.com/module.php?module=xml_sitemaps_show_sitemap or if you want the sitemap to look friendly ( btw ask.com will only accept a friendly sitemap ending in .xml ) , you just have to go into Admin->Configuration->XmlSitemaps and enable "Sitemap Friendly URL", and if you do that then you have to put the following lines in your .htaccess somewhere before the line "##### URL Method 2 ("Clean" URLs) Begin #####" :

RewriteRule ^sitemapindex.xml module.php?module=xml_sitemaps_show_sitemap [L]
RewriteRule ^sitemap-([a-zA-Z0-9]+).xml module.php?module=xml_sitemaps_show_sitemap&i=$1 [L]

Here is how the index looks on a site with sitemap friendly urls enabled: http://sapa.ro/sitemapindex.xml

There are other configuration options in there, you can set the maximum number of stories to put in a sitemap, and you can chose whether to ping any of the three search engines supported. You can also set your yahoo.com key in there if you want to ping yahoo.

That's it! Happy Sitemapping! and as always ... let me know how it works in the comments.

These icons link to social bookmarking sites where readers can share and discover new web pages.
  • Digg
  • Reddit
  • del.icio.us
  • Slashdot
  • Spurl
  • StumbleUpon
  • Furl
  • DZone
  • Netscape
  • NewsVine
  • Technorati
  • YahooMyWeb
  • Simpy
If you enjoyed this post, you should subscribe to my full RSS Feeds

RSS feed | Trackback URI

62 Comments »

collapse thread Comment by Pradeep Sethi Subscribed to comments via email
2007-08-26 00:31:51

Hi

I am trying this in 9.7 and am getting the following error:

Fatal error: Unknown function: clear_cache() in /home/delhirea/public_html/delhirealestate/modules/xml_sitemaps/xml_sitemaps_install.php on line 5

 
collapse thread Comment by Pradeep Sethi Subscribed to comments via email
2007-08-26 00:36:16

I could install it after commenting that line. Thanks

 
collapse thread Comment by Mihai
2007-08-27 13:29:49

I guess they removed or renamed that function. That was useful only when you enable/re enable the module. So you should be fine without it. Maybe just delete everything in the cache/ folder after you enable the module.

 
collapse thread Comment by Jody D Subscribed to comments via email
2007-08-30 16:26:00

Hey there, great tools, have one question though: I installed and activated the module but when I looked at the sitemap http://www.ebenta.com/module.php?module=xml_sitemaps_show_sitemap&i=0, i found out that the date shown was invalid. I am getting c on it. What is wrong with the installation? Is this by default the settings? How can I correct it to show the right date format (sample: 2005-02-21T18:00:15+00:00)? Any help is appreciated.

Here is the url of my sitemap: http://www.ebenta.com/module.php?module=xml_sitemaps_show_sitemap

Thanks.

 
collapse thread Comment by Mihai
2007-08-30 17:27:54

The problem is the php version that you use. Most likely you have php 4.x.
The module uses the date function with the “c” format, which was added in php 5 .
So either switch to php 5 or modify the module, and replace date(”c” with something like date(”Y-m-dTH:i:s” …
Switching to php 5 is recommended since the php development team will drop support for php 4 soon

 
collapse thread Comment by Jody D Subscribed to comments via email
2007-08-30 18:03:36

Great! I figured it out. I am running 4.x. Fixed it. Thanks!

 
collapse thread Comment by Guy
2007-09-04 12:23:22

Looks great! I ran into a couple of problems:
1. It looks like it’s already been figured out: PHP5 is required
2. xml_sitemaps_main.php:Line 88 should change UNIX_TIMESTAMPT to UNIX_TIMESTAMP
3. xml_sitemaps_install.php:Line 22 should change ^sitemap-([0-9]+).xml to ^sitemap-([a-zA-Z0-9]+).xml
4. Categories aren’t being escaped to their safe names. I have some categories with ampersands (&) that don’t display when following the sitemap link
5. The site’s URLMethod configuration is being ignored. I’m using URLMethod 1 (non-clean URLS), but the sitemap generated uses clean URLs

collapse thread Comment by Mihai
2007-09-04 13:37:10

thanks for the feedback. I’ll release a fix soon. If you have a patch feel free to submit it.

 
 
collapse thread Comment by dhirendra patel Subscribed to comments via email
2007-09-04 17:54:22

Hi

I am trying this in 9.7 and am getting the following error:

Fatal error: Unknown function: clear_cache() in /home/delhirea/public_html/delhirealestate/modules/xml_sitemaps/xml_sitemaps_install.php on line 5

 
collapse thread Comment by Mihai
2007-09-04 18:01:13

It would be a good idea to read the other comments before you post…someone may have found a fix…

 
collapse thread Comment by Daniel
2007-09-06 07:17:51

I am getting the following error when I go to my sitemap, is there a way to fix this ?

Warning
: You have an error in your SQL syntax. Check the manual that corresponds to your MySQL server version for the right syntax to use near ’select link_modified from linka_links where link_status!=’disca in
/home/nzweblink/linka/libs/db.php
on line
207

http://www.linka.co.nz/module.php?module=xml_sitemaps_show_sitemap&i=0

1970-01-01T12:00:00+12:00

http://www.linka.co.nz/sitemap-pages.xml

collapse thread Comment by Mihai
2007-09-06 10:57:54

It may be a problem with your php configuration options. Maybe you have magic_quotes_gpc enabled and the data may get double escaped…
It would help if you could echo that sql before the error so that we have the whole sql query too look at and debug.

 
 
collapse thread Comment by Daniel
2007-09-07 09:47:55

How do you mean “it would help if you could echo that sql before the error”, sorry I don’t understand.

Click here to go to the page where the error is

 
collapse thread Comment by Mihai
2007-09-07 11:24:34

look xml_sitemaps_main.php, at line 17 there should be this:
$sql = “SELECT count(*) FROM “.table_links.” WHERE link_status!=’discard’”;
look in the database and see if the link_status field exists…
try to run this query :
select link_modified from linka_links where link_status!=’discard’
in phpmyadmin or the mysql client manually and see what result you get …

collapse thread Comment by Daniel
2007-09-08 04:09:23

when I run the sql query: SELECT link_modified FROM linka_links WHERE link_status != ‘discard’

The result shows link_modified with a number below eg (20070712192533), running the query returns a normal result with no errors

The link_status field does exist

 
collapse thread Comment by Mihai
2007-09-08 10:08:27

Did you check that line ( 17 ) in xml_sitemaps_main.php ?
How does it look ?

collapse thread Comment by Daniel
2007-09-08 10:12:10

This is how it looks:

$sql = “SELECT count(*) FROM “.table_links.” WHERE link_status!=’discard’”;

 
 
 
collapse thread Comment by Raj Subscribed to comments via email
2007-09-17 17:44:53

Please tell me solution for this error

Warning
: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ‘XmlSitemaps_Links_per_sitemap ) l’ at line 1 in
/home/tagrin/public_html/libs/db.php
on line
233

collapse thread Comment by Albert F A Matthews
2008-01-04 06:48:22

To what you wrote ok you have not set up the sitemap in the admin section
Goto
Pligg beta 9 Home » Admin » Configure Pligg Beta 9
and if you do this it will remove all of the sql database problems like

Warning
: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ‘XmlSitemaps_Links_per_sitemap ) l’ at line 1 in
/home/tagrin/public_html/libs/db.php
on line
233

just a head up and im using the new one out Pligg_Beta_9.8.2

hope this helps,
Albert F A Matthews
http://www.wedigg.co.uk

 
 
collapse thread Comment by Mihai
2007-09-18 17:12:56

Have you modified this the module in any way? What php and mysql version do you you run?

 
collapse thread Comment by Raj Subscribed to comments via email
2007-09-18 19:42:40

No i havent modified anything… iam using php 4.4 and mysql 4

collapse thread Comment by Guy
2007-09-18 21:42:11

This is a PHP 4 thing. You need to use PHP 5, but the MySQL version should be okay.

 
 
collapse thread Comment by Mihai
2007-09-18 21:04:30

It was reported before that this module has some problems on php 4 and this may be one of your problems. I also haven’t tested this on mysql 4 so that may be the other problem.

 
collapse thread Comment by Daniel
2007-09-19 00:50:25

Hi Mihai,

I still haven’t figuered out why I am getting the error I am from the above comments by me “Daniel 2007-09-06 07:17:51″

 
collapse thread Comment by Raj Subscribed to comments via email
2007-09-19 15:24:33

Why dont you release for php 4. I think most of people use version 4.

 
collapse thread Comment by Mihai
2007-09-19 19:55:27

I don’t currently have a host running php 4. And you should really upgrade anywany. The php development team announced a few months ago that they will drop support for php 4.

 
collapse thread Comment by Mihai
2007-09-19 19:56:16

@Daniel : Are you running php 4?
some other reported it’s not working on php 4 so you may have the same problem too.

 
collapse thread Comment by Travel Voter Subscribed to comments via email
2007-09-20 07:03:24

http://travelvoter.com/module.php?module=xml_sitemaps_show_sitemap
This works very well with 9.8.
Thanks!

P.S. We couldn’t get the friendly sitemap url.

collapse thread Comment by dario
2007-09-22 18:37:45

Your sitemap doesn’t, and mine doesn’t; i’m using pligg 9.81, and using php 5

collapse thread Comment by Mihai
2007-09-22 18:45:06

My sitemap doesn’t do laundry either….
anyway thanks for the great detailed feedback :)

 
 
 
collapse thread Comment by Mihai
2007-09-20 07:19:27

you have to use the htaccess rules I described in this post.
Also you might want to upgrade to php5 or stay tuned for an update of this plugin that may make this work on php 4 ( the last mod date doesn’t work on php 4 )

 
collapse thread Comment by Nissa
2007-10-02 18:13:15

Hi mihai..
I cant install your sitemap module. What i do is: extract .tar file and upload to root modules folder. And i go to admin and Module Management. But the xml_sitemaps not show there. For your information i use pligg 9.8. its rite the step what i do?

collapse thread Comment by Mihai
2007-10-02 20:43:06

this problem may be related with the call to clear_cache function in xml_sitemaps_install.php. Editing that file and removing that call should let you install the module

collapse thread Comment by Nissa
2007-10-02 22:08:01

Many thank Mihai.. Your sitemap great working now. i just delete clear_cache in xml_sitemaps_install.php. Btw after install… did i must/need to add again clear_cache in xml_sitemaps_install.php ?

 
 
 
collapse thread Comment by jatboy Subscribed to comments via email
2007-10-09 05:29:57

Has anyone made it work on 9.8.2? I don’t even see it in the Available Modules.

 
collapse thread Comment by hostmenow
2007-10-14 20:18:26

You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ‘XmlSitemaps_Links_per_sitemap ) l’ at line 1 in

This error I am getting . Please tell me how can i rectify this . The php version is proper . Is it due to my sql version or what .

Also as the number of postings are more than 5000 in case multiple files are created do we have to give write permissions in some files or what.

 
collapse thread Comment by Amit
2007-10-16 06:12:29

Getting an error:

Fatal error: Call to undefined function clear_cache() in /home/desitoob/public_html/modules/xml_sitemaps/xml_sitemaps_install.php on line 5

Using Pigg v9.8

Let me know. Thanks.

collapse thread Comment by Isaac Subscribed to comments via email
2007-10-31 15:05:39

Go to line 5 of the file and remove clear_cache(). As easy as 1,2,3.

collapse thread Comment by Albert Matthews
2007-12-31 02:44:50

Thank you ever so much Isaac,

Seems to be a popular problem that onebut i read thru and go to your post and done what you said and bame all done and installed thanks

great advice and thanks
Mihai Secasiu PatchLog.com

Warmest regards
Albert F A Matthews

 
 
 
collapse thread Comment by wwwoliondorcom Subscribed to comments via email
2007-10-27 17:54:19

Sorry, me again, because I got 2 errors on Google sitemap.

I have enabled clean sitemap url and given this sitemap address to Google sitemap:
http://www.MY-WEBSITE.com/sitemapindex.xml

at http://www.MY-WEBSITE.com/sitemapindex.xml I have

http://www.MY-WEBSITE.com/sitemap-0.xml
2007-10-27T13:34:02-07:00

http://www.MY-WEBSITE.com/sitemap-pages.xml

The errors on Google sitemap are:

http://www.MY-WEBSITE.com/sitemap-0.xml
Nested indexing
This URL refers to a Sitemap index, not a Sitemap. Please remove it and resubmit.

http://www.MY-WEBSITE.com/sitemap-pages.xml
Nested indexing
This URL refers to a Sitemap index, not a Sitemap. Please remove it and resubmit.

Please tell me if you see what is the problem.

Thanks.

 
collapse thread Comment by hemant Subscribed to comments via email
2007-11-30 17:51:55

Hi
I get this error:

Fatal error: Call to undefined function: clear_cache() in /home/content/r/u/b/ruberr2002/html/modules/xml_sitemaps/xml_sitemaps_install.php on line 5

Please advise

 
collapse thread Comment by hemant Subscribed to comments via email
2007-11-30 17:53:16

I run 9.6 pligg & get this message:
Fatal error: Call to undefined function: clear_cache() in /home/content/r/u/b/ruberr2002/html/modules/xml_sitemaps/xml_sitemaps_install.php on line 5

Please advise. Thanks

 
collapse thread Comment by Digispro
2007-12-06 19:53:46

Hi,

I’m not getting an empty sitemap.

Here is the url

http://www.awomanis.com/awi/module.php?module=xml_sitemaps_show_sitemap

I think I’m not doing something right.

 
collapse thread Comment by Thomas Herold
2007-12-11 01:55:32

I am new to this. I have uploaded and extracted the folder inside the module folder, however it does not show up in my admin control panel.

Can you help me out?

Thomas

 
collapse thread Comment by Nick Subscribed to comments via email
2007-12-30 18:25:33

Can you us some best practice thoughts on the number of links per sitemap? I have over 100,000 articles, and I want to index them all, is 50,000 a good number?

 
collapse thread Comment by Nick Subscribed to comments via email
2007-12-30 18:26:42

Ohh, and did I mention how awesome this is? I was working on something similar to this all morning, and then I found your module, and saved me time. Thanks for the awesome mod, and the support!

 
collapse thread Comment by zorba33 Subscribed to comments via email
2008-01-19 14:57:15

Hello,

Everything is ok , xmlsitemap is running on my 9.8.2 pligg website.

The only problem is that the sitemap I get is empty.

Can you help?

BR

 
collapse thread Comment by zorba33 Subscribed to comments via email
2008-01-20 16:06:55

hello,

I am getting an empty sitemap.
http://88.191.13.182/module.php?module=xml_sitemaps_show_sitemap

Can you help?

 
collapse thread Comment by Nick Subscribed to comments via email
2008-01-21 16:53:16

Hey BR,

It looks like in the config you are using friendly URLs. Take off the friendly URLs and see if it works that way, chances are its working great. There are some mods that you need to do to make friendly URL’s work. but it all works perfectly. Its an awesome mod.

 
collapse thread Comment by DR
2008-02-08 16:30:49

Anyone know something about this error?

Warning: You have an error in your SQL syntax. Check the manual that corresponds to your MySQL server version for the right syntax to use near ’select link_modified from pligg_links where link_status!=’disca in /homepages/28/d123456/htdocs/homepages/domain/libs/db.php on line 207

 
collapse thread Comment by albert matthews
2008-02-17 06:19:23

where can i get a working XML Sitemaps that works 9.9

i had the other one but, im not sure if the old one is working ??

help “” advise “”

 
collapse thread Comment by anil
2008-02-22 09:48:15

Gr8 module..i really love 2 use it but only a prob i am facing is
http://www.jeqq.com/sitemap-pages.xml

error with only ths help any help guys

 
collapse thread Comment by Andres
2008-03-03 15:52:25
 
collapse thread Comment by whatnwhen Subscribed to comments via email
2008-03-23 10:17:11

http://www.whatnwhen.in/sitemapindex.xml

I am getting only 2 urls and thats two….don’t know from where its pulling, can you help on this please…….

 
collapse thread Comment by DevCore
2008-04-13 05:48:26

I have got PHP5 installed, but either way this module has this error:

Fatal error: Call to undefined function clear_cache() in /home/_______/public_html/modules/xml_sitemaps/xml_sitemaps_install.php on line 5

I am running Pligg’s latest version.

collapse thread Comment by Mihai
2008-04-13 08:39:09

just remove line 5 in xml_sitemaps_install.php

 
 
Name (required)
E-mail (required - never shown publicly)
URI
Subscribe to comments via email
Your Comment (smaller size | larger size)
You may use <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong> in your comment.

Trackback responses to this post


Creative Commons License
This work is licensed under a Creative Commons Attribution-Share Alike 3.0 License

Technology Blogs - Blog Top Sites Search For Blogs, Submit Blogs, The Ultimate Blog Directory Blogarama - The Blog Directory 5starsblog Computers Blogs - Blog Flare blog search directory gob BlogHop