Bug Pending upgrade: DragonByte Credits

[fly]

I'm seeing the following error when initially installing DB Credits:

Pending upgrade: DragonByte Credits - s3://ufdata/dbtechCreditsUpgrade.lock

Now, obviously I push the /data folder to Amazon S3, but that file doesn't seem to exist there.

 

[Fillip H.]

This is not a bug. If PHP detects that file exists, that message will show. This is something you will need to resolve with your system administrator.

 

[fly]

I am the system admin, and the file does not exist.

edit: If it helps, when I turn off S3 redirection in the xF config file, the error goes away.

 

[fly]

And to expand on that, here's the ExternalDataPath from my config.php with sensitive data removed:

// Push to S3
require_once 'Zend/Service/Amazon/S3.php';
$s3 = new Zend_Service_Amazon_S3('xxxxxxxxxxxxxxx', 'xxxxxxxxxxxxxxxxxxxxxx');
$s3->registerStreamWrapper("s3");
$config['externalDataPath'] = 's3://data.uselessforums.com';
$config['externalDataUrl'] = 'https://data.uselessforums.com';

And here is me verifying that the file isn't there:

[ec2-user@ufDB html]$ aws s3 rm s3://data.uselessforums.com/dbtechCreditsUpgrade.lock
fatal error: An error occurred (404) when calling the HeadObject operation: Key "dbtechCreditsUpgrade.lock" does not exist


You might notice that the S3 bucket has changed. I recreated it with a different name to ensure there wasn't some strange problem with the bucket itself.

 

[Fillip H.]

Sorry, I wasn’t clear: in order to check if that file exists, we use a built-in PHP function: PHP: file_exists - Manual - this function is returning true when checking if that file exists on your system.

Why that is, I cannot say, but the problem is with your system and/or with how PHP on your system handles checking whether remote files exist.

This is something you need to resolve with your system, as the issue is not in our code.

Hopefully that clarified things for you

 

[fly]

Doesn't that seem odd though? file_exists is pretty simple, so it's almost certainly not a problem with PHP. I see

XenForo_Helper_File::getExternalDataPath()

in your code. I just did a grep for that, and dbCredits isn't the only thing that uses that. I see it used in the default Xenforo files, as well as two other addons I'm using. Everything else seems to be working without issue.

 

[Fillip H.]

The only thing I could do would be to move the file to the internal_data folder, which presumably is not synchronised (it shouldn't be, at any case).

It's not a simple change so I wouldn't have a timeline for this, but in the meanwhile you can turn on debug mode and disable the relevant code event listener from the product - I don't have the information in front of me at the moment but the listener should have a description containing something like "upgrade notice".

 

[fly]

Yes, in my case, the only thing synced to S3 in internal_data is attachment data (using an xfrocks addon). I'll disable that listener, thanks. Is this error actually affecting anything, or is it a bit of a red herring?

 

[Fillip H.]

Actually I just realised that won't work, since the file_exists check happens at the top of every event listener. What it does is prevent every listener from running, so it would mean no integration actually works.

The reason why I set this up is because pending upgrades can cause DB errors, and it's error prevention. I would instead go back to the idea of looking into why file_exists checks return incorrect results for your s3 URL.

 

[fly]

Short story: Please make sure that moving the file check to internal_data is on your roadmap.

Long story: So I sorta worked around this by installing the [bd] Data Storage addon instead of simply using the xF streaming feature. However, my server ran out of space the other day due to an extremely large log file. Turns out this storage addon writes an error if a key (what Amazon calls an S3 file) doesn't exist. So every time this addon checks for that file, the data storage addon writes an error because it doesn't exist.

I reached out to xfrocks, the author of that plugin, asking if logging of 'key not found' could be removed. He said that he would work on it, but also included this, which matches up with your idea:

Sorry for your problem. We can certainly make the change to omit logging even in the event of an error. However, I think this kind of lock file should be in internal_data instead of data. How often is the check? It may cause site performance issue because S3 api is quite slow.

And indeed, I have been noticing some performance issues that I just haven't had a chance to track down.

 

[Fillip H.]

Alright, I'll see if I can get something set up. I'm not 100% sure if I'll be able to hotfix all existing products without major hassles but I'll see what happens once I start the work on it

 

[Fillip H.]

Update: Turns out this could be changed using search & replace without trouble, and hotfixing also worked as I'd hoped, so if you re-download all your XF1 mods from us and re-upload the files, the checks should now point to the internal file.

 

[fly]

Safe to assume that it's just file uploads and I don't need to reimport the XML?


edit: Seems like it, once I delete the .lock file.

 

[Fillip H.]

I believe so

 

[DragonByte Technologies]

Hello fly,

This ticket has now been closed with the status Fixed.

We hope your issue or question has been addressed to your satisfaction. If not, please feel free to re-open it by clicking this link.

If you have any further issues or questions, please feel free to start a new support ticket via the button at the top of every page.

Thank you!