Bug Auto Sitemap Generation Problem

[CharlieDelta]

I have the auto sitemap generation setup for every 1 day.
I just realized I had not been getting any emails lately so I check the log and the last generation was on September 17. I run it manually with no issues.
Seems to be an issue with the auto generator.

 

[Fillip H.]

I checked your error_log file and found some error messages related to the old directories being left in place. I've deleted those directories and removed the error_log file, but we'll just have to see whether it generates correctly next time.

 

[CharlieDelta]

So it has been three days and it has not auto generated at all.

 

[Fillip H.]

Are there any new entries in the error_log pertaining to DBSEO?

 

[CharlieDelta]

There are no recent entries in the error log of any kind. The last entry is dated September 22

 

[Fillip H.]

The only thing I can think of is that not enough users were active on your site at the time the sitemap is set to be generated, so by the time all other scheduled tasks were processed, an hour has gone by, thus skipping that day in the generation.

Could you set it so that it is scheduled to be processed the next full hour (so if the server and your account was set to GMT, it would be 23:00) and observe?

 

[CharlieDelta]

I set it for one hour as you suggested. There was three users active at the time it was set to run and I also was on the site on another computer as a guest.
After roughly 2 minutes the cron was pushed off for 30 minutes. I then observed at this time and there was again three users active and I was also active as a guest. It then got pushed off exactly 24 hours and did not run.

I checked the error log and there is no recent entries.

 

[Fillip H.]

24 hours? That doesn't make any sense, it's supposed to be set to run every hour to check when it's supposed to actually run based on the schedule...

 

[CharlieDelta]

Yes. It jumped to the next day. You can see for yourself.

 

[Fillip H.]

Could you re-create an FTP account for me, and let me know what time zone the server is set to so I can look into why this is happening?

 

[Fillip H.]

I tried setting the sitemap generation to run at the same hour as what it currently was, and I was able to generate it via the Scheduled Task Manager.

Tomorrow I'll try again, but using the cron.php file via the front-end and see if that makes a difference.

 

[bzcomputers]

I too have to agree there still isn't something quite right with the auto sitemap generation. I know I've reported issues in the past with it and it seems to be better now than it was (at least for me), but it still has random issues in running.

I know for me the first issue I noticed is that upon a new version upgrade of DBSEO it would always skip the first day of auto generation. I also have the sitemap set to run once a day as CharlieDelta does. Although my issues don't seem to be as bad as his, this is my sitemap generation log from the last 60 days.

Sitemap ran on:
Aug. 16, 17, 19, 20, 21, 23, 25, 26, 27, 29, 30
Sept. 1, 3, 5, 7, 8, 10, 11, 13, 15, 16, 18, 20, 22, 23, 25, 26, 27, 29, 30
Oct. 2, 4, 6, 8, 9, 10, 11, 12, 14, 16

Sitemap email reports received are identical to the log dates above, so I can assume it's not just an issue of the log not being created or email being sent but is truely an issue with the sitemap cron job not running on days it should.

 

[Fillip H.]

To eliminate the possibility that it's not just a lack of activity that's causing the sitemap to get skipped, could you both try using the sitemapcron.php file from the /do_not_upload folder?

Turn off the sitemap in the DBSEO CP, put that file somewhere outside of public_html, then add it to crontab.

For a CentOS based system, an example entry into crontab -e would be:

0 4 * * * /usr/bin/php /home/dbtech/sitemapcron.php 2>&1 > /dev/null

This would run the script every night at 4 AM server time. To test that this cron job is working, manually run the command

/usr/bin/php /home/dbtech/sitemapcron.php

and observe the output.

(Replace /usr/bin/php if that's not the path to your php executable, and replace the path to the script with the actual path.)

 

[bzcomputers]

Did as requested above. Only change I had to make was to modify the sitemapcron.php file with the actual forum directory on line 20.

When attempting to run manually I get:

[root@host /usr/bin]# /usr/bin/php /xxxx/xxxxxx/cron/sitemapcron.php
Status: 301 Moved Permanently
X-Powered-By: PHP/5.4.27
Location: Forum - CRUISIN
Content-type: text/html

Not sure what it is referring to that is giving the 301.


I'm almost positive that lack of activity is not the issue on my site. At the time the cron scheduled to run the average users online is around 40-50.

 

[Fillip H.]

That's quite strange, I have no idea why anything outside of public_html would create a 301 redirect... Do you have a .htaccess file in /home/cruisin that may contain a redirect?

 

[bzcomputers]

That's quite strange, I have no idea why anything outside of public_html would create a 301 redirect... Do you have a .htaccess file in /home/cruisin that may contain a redirect?

No, thats why I was a little confused by it too.

 

[CharlieDelta]

Sorry I don't have access to do this, or at least I think I don't. I am on a shared server. I am also not versed on crontab and the like so if I can do this please by all means let me know how and I will make it happen.

 

[Fillip H.]

If you have cPanel access, you can do it from there, via the Cron jobs interface. You'd set it to run like so: https://www.dropbox.com/s/v9u24e36pw39zn0/Screenshot 2014-10-18 15.34.32.png?dl=0

 

[CharlieDelta]

Perfect! I can do that no problem. I am lost when it comes to the file path however. How do I ensure I have the correct path? That I need to learn.

 

[Fillip H.]

You can find your path by inspecting the "DOCUMENT_ROOT" variable in the View PHP Info page. That will be your public_html path, so you should put this file outside that path.