Bug DBSEO Stop Words Not Working Properly

[neounix]

HI. We currently are seeing a problem with Stop Words.

For example:

http://my.server.com/help-with-splitting-and-rearranging/shell-programming-and-scripting/t238897/post302866875/#post302866875

We see the "and" stop word in both [thread-title] and [forum-title] (above)

FYI, the format for this test is:

http://my.server.com/[thread_title]/[forum_title]/t[thread_id]/post[post_id]/#post[post_id]

 

[Fillip H.]

What's the name of the URL Format setting you edited to produce that URL? I'll need to know so that I can look at the specific code that generates that URL.

 

[neounix]

Here you go..

http://my.server.com/showthread.php?p=302866875#post302866875

 

[Fillip H.]

No, I need the setting name in the DBSEO AdminCP - not the un-SEO'd URL

 

[neounix]

I assume it is:

Custom Thread Go To Post URL

 

[Fillip H.]

I'm unable to replicate any issues with that particular format or indeed any other thread format. Can you please double-check your settings and ensure you gave me the correct format name based on the format setting you showed me in post #1 ?

 

[neounix]

OK. I'll get back to you on this later today.

 

[neounix]

See below (next post)

 

[neounix]

Yes, I just confirmed the Stop Words are not working.

Original URL (Before DBSEO)

http://myserver.domain.com/showthread.php?t=238897

Turn on DBSEO, URL is now:

http://myserver.domain.com/help-with-splitting-and-rearranging/shell-programming-and-scripting/t238897/

The Stop Words are in the URL.

Settings:

URL Rewrite Settings -> Forum URLs -> Thread URL - Custom

Custom Thread URL

[thread_title]/[forum_title]/t[thread_id]/

You can see the URL after DBSEO matches perfectly the Custom Thread URL format, but the Stop Words are in the URL.

Cheers.

 

[neounix]

FYI and Update:

We have checked and the Stop Words are not being filtered out for either showthread.php or showpost.php URL.

In fact, we cannot find where Stop Words are being filtered out on any URLs and the setting is definitely turned on at our site.

 

[Fillip H.]

Could you please create and PM me with a temporary FTP and AdminCP account?

For security reasons, we recommend you create a new FTP account only for DBTech support, then disable or delete it after we have both confirmed the issue has been solved and there are no further issues.

The same applies to AdminCP accounts; they should ideally be temporary accounts created for us only. If we have created an account on your site already, you can optionally boost that account to Administrator and then de-admin this account once the issue has been solved.

If you use a .htaccess password protection for your AdminCP directory, it is recommended that you create a new authorised user for DBTech and remove this user once the issue has been solved.

Please test any temporary accounts you create to ensure that the FTP account has access to the forum files, and that the AdminCP account can access the administrative controls for the product we are assisting you with.

Ensuring this is all in order before submitting the information will significantly speed up the process of assisting you. We will alert you via PM if there's any issues with the login information you have provided.

When sending the PM, for your security you should also un-tick the "Save a copy in my Sent Items folder" checkbox. When the access details have been received, we will delete the PM from our inbox. Ensuring you have not kept a copy of the PM reduces the risk of security breaches.

Thank you for helping us debug our products and allowing us to assist you, we appreciate it

 

[neounix]

(done)

 

[Fillip H.]

Upon further investigation, I've discovered that this is working as intended.

The setting "Maximum URL Keywords" also controls the minimum length of the URL. help-with-splitting-and-rearranging is exactly 5 words long. Removing "and" would bring this down beneath the configured limit.

Changing this behaviour to always filter stopwords will have negative impact on legacy vBSEO-created URLs.

I will update the description of that setting for Beta 9 to clarify that stopwords will not be removed if not enough keywords are found in the URL.

 

[Fillip H.]

Update: The setting has been renamed URL Length Target and the description now reads

You can use this setting to control the maximum number of keywords that will be used when generating your URLs.<br />
<br />
If more keywords are found, they will be trimmed off the end of your URL.<br />
Not enough keywords are found, stopwords will only be filtered after the URL has reached the configured length.<br />
<br />
Recommended value is either <code>0</code> (default) or a value between 5 and 7.

 

[neounix]

Sorry, I don't understand "this is the intended behavior".

The title of the thread we are testing is:

Help with splitting and rearranging a field in awk

DBSEO is using the first five words as keywords and hence using two stop words instead of using two other important "non stop words" (good keywords).

The URL should be:

help-splitting-arranging-field-awk

That was the way VBSEO did it as well. It removed the stop words .... and even with a five stop words set, there are five "non stop" words to use.

Hence, DBSEO is not working as it should, in my view.

 

[Fillip H.]

You're actually correct, I was mis-reading what was going on. I've hotfixed the file on your site and reverted the setting to 5.

Can you confirm it works correctly on other threads as well?

 

[Fillip H.]

I'll revert the setting name / description as the correct functionality does not reflect the new description.

 

[neounix]

Cheers and thanks for looking into this.

I checked, and it seems not to be working correctly (or how I expect).

For example, this is correct:

Title: Disk Unable to Boot

DBSEO URL: disk-unable-book

But this one seems incorrect:

Title: What does this mean?

DBSEO: mean

I think I have the setting set to "5" in this case, the token should be:

what-does-this-mean

... and in another example:

Title: Find the smallest block

DBSEO: block

I think it should be:

find-smallest-block

(so I remove "find" and "smallest" from stop word list and now it is OK.

So, the issue "on the table" seem to be "what do we do when after the stop words are all removed and there are not enough keywords.

Then, I create a thread with "only stop words"

TItle: What Do We Do

DBSEO: what-do-we-do

So, this is correct, since all are stop words, it is working properly.

What I think needs to happen when there is only one keyword, stop words should be added.

For example:

Title: What Do We Do Unix

DBSEO unix

I think maybe it should be:

what-do-we-do-unix

.. since we have it set to five... it should be okay to include the stop words because we do not lose any keywords.



.

 

[neounix]

Update:

OK.. I simply replaced the very big list of stop words that come default with DBSEO, with the original list of stop words that was used for (my old version of) VBSEO, and all seems OK.

a
an
and
are
as
at
be
by
for
from
in
is
it
of
on
or
that
the
this
to
was
which
with

 

[neounix]

Hmm.. After testing, I'm not sure what is best..... seems this might be an area to consider some more options for the admin.

Either way, I really like the flexibility of DBSEO over VBSEO, and the support is much better !!!

Thank you for working so hard on this.