Bug InvalidArgumentException: Failed to convert to JSON. Error: Malformed UTF-8 characters, possibly incorrectly encoded [raw_data]

[wmtech]

Found this error in our error log. Happened when querying the api for purchased products. However the api query seems to have returned just fine, so just for your information.

InvalidArgumentException: Failed to convert to JSON. Error: Malformed UTF-8 characters, possibly incorrectly encoded [raw_data] src/XF/Mvc/Entity/Entity.php:740
Generated by: Themehouse Jan 14, 2020 at 9:24 AM

Stack trace
#0 src/XF/Mvc/Entity/Entity.php(618): XF\Mvc\Entity\Entity->_castValueToType(Array, 'raw_data', 65552, Array)
#1 src/XF/Mvc/Entity/Entity.php(548): XF\Mvc\Entity\Entity->set('raw_data', Array)
#2 src/addons/DBTech/eCommerce/ApiRequestLog/Logger.php(43): XF\Mvc\Entity\Entity->__set('raw_data', Array)
#3 src/addons/DBTech/eCommerce/XF/Logger.php(55): DBTech\eCommerce\ApiRequestLog\Logger->log('6zac7txddx9vaZT...', '/api/dbtech-eco...', false, 'http://localhos...', 'localhost', 'XenForo', '2010671', Array, true)
#4 src/addons/DBTech/eCommerce/Api/Controller/AbstractLoggableEndpoint.php(47): DBTech\eCommerce\XF\Logger->logDbtechEcommerceApiRequest('6zac7txddx9vaZT...', '/api/dbtech-eco...', false, 'http://localhos...', 'localhost', 'XenForo', '2010671', Array)
#5 src/addons/DBTech/eCommerce/Api/Controller/AbstractLoggableEndpoint.php(29): DBTech\eCommerce\Api\Controller\AbstractLoggableEndpoint->logApiRequest()
#6 src/addons/DBTech/eCommerce/Api/Controller/Products.php(21): DBTech\eCommerce\Api\Controller\AbstractLoggableEndpoint->preDispatchController('Getpurchased', Object(XF\Mvc\ParameterBag))
#7 src/XF/Api/Controller/AbstractController.php(22): DBTech\eCommerce\Api\Controller\Products->preDispatchController('Getpurchased', Object(XF\Mvc\ParameterBag))
#8 src/XF/Mvc/Controller.php(123): XF\Api\Controller\AbstractController->preDispatchType('Getpurchased', Object(XF\Mvc\ParameterBag))
#9 src/XF/Mvc/Dispatcher.php(349): XF\Mvc\Controller->preDispatch('Getpurchased', Object(XF\Mvc\ParameterBag))
#10 src/XF/Api/Mvc/Dispatcher.php(31): XF\Mvc\Dispatcher->dispatchClass('DBTech\\eCommerc...', 'Getpurchased', Object(XF\Api\Mvc\RouteMatch), Object(DBTech\eCommerce\Api\Controller\Products), NULL)
#11 src/XF/Mvc/Dispatcher.php(113): XF\Api\Mvc\Dispatcher->dispatchFromMatch(Object(XF\Api\Mvc\RouteMatch), Object(DBTech\eCommerce\Api\Controller\Products), NULL)
#12 src/XF/Mvc/Dispatcher.php(55): XF\Mvc\Dispatcher->dispatchLoop(Object(XF\Api\Mvc\RouteMatch))
#13 src/XF/App.php(2184): XF\Mvc\Dispatcher->run()
#14 src/XF.php(391): XF\App->run()
#15 index.php(16): XF::runApp('XF\\Api\\App')
#16 {main}

-------------

Previous InvalidArgumentException: Failed to convert to JSON. Error: Malformed UTF-8 characters, possibly incorrectly encoded - src/XF/Mvc/Entity/ValueFormatter.php:164
#0 src/XF/Mvc/Entity/Entity.php(736): XF\Mvc\Entity\ValueFormatter->castValueToType(Array, 65552, Array)
#1 src/XF/Mvc/Entity/Entity.php(618): XF\Mvc\Entity\Entity->_castValueToType(Array, 'raw_data', 65552, Array)
#2 src/XF/Mvc/Entity/Entity.php(548): XF\Mvc\Entity\Entity->set('raw_data', Array)
#3 src/addons/DBTech/eCommerce/ApiRequestLog/Logger.php(43): XF\Mvc\Entity\Entity->__set('raw_data', Array)
#4 src/addons/DBTech/eCommerce/XF/Logger.php(55): DBTech\eCommerce\ApiRequestLog\Logger->log('6zac7txddx9vaZT...', '/api/dbtech-eco...', false, 'http://localhos...', 'localhost', 'XenForo', '2010671', Array, true)
#5 src/addons/DBTech/eCommerce/Api/Controller/AbstractLoggableEndpoint.php(47): DBTech\eCommerce\XF\Logger->logDbtechEcommerceApiRequest('6zac7txddx9vaZT...', '/api/dbtech-eco...', false, 'http://localhos...', 'localhost', 'XenForo', '2010671', Array)
#6 src/addons/DBTech/eCommerce/Api/Controller/AbstractLoggableEndpoint.php(29): DBTech\eCommerce\Api\Controller\AbstractLoggableEndpoint->logApiRequest()
#7 src/addons/DBTech/eCommerce/Api/Controller/Products.php(21): DBTech\eCommerce\Api\Controller\AbstractLoggableEndpoint->preDispatchController('Getpurchased', Object(XF\Mvc\ParameterBag))
#8 src/XF/Api/Controller/AbstractController.php(22): DBTech\eCommerce\Api\Controller\Products->preDispatchController('Getpurchased', Object(XF\Mvc\ParameterBag))
#9 src/XF/Mvc/Controller.php(123): XF\Api\Controller\AbstractController->preDispatchType('Getpurchased', Object(XF\Mvc\ParameterBag))
#10 src/XF/Mvc/Dispatcher.php(349): XF\Mvc\Controller->preDispatch('Getpurchased', Object(XF\Mvc\ParameterBag))
#11 src/XF/Api/Mvc/Dispatcher.php(31): XF\Mvc\Dispatcher->dispatchClass('DBTech\\eCommerc...', 'Getpurchased', Object(XF\Api\Mvc\RouteMatch), Object(DBTech\eCommerce\Api\Controller\Products), NULL)
#12 src/XF/Mvc/Dispatcher.php(113): XF\Api\Mvc\Dispatcher->dispatchFromMatch(Object(XF\Api\Mvc\RouteMatch), Object(DBTech\eCommerce\Api\Controller\Products), NULL)
#13 src/XF/Mvc/Dispatcher.php(55): XF\Mvc\Dispatcher->dispatchLoop(Object(XF\Api\Mvc\RouteMatch))
#14 src/XF/App.php(2184): XF\Mvc\Dispatcher->run()
#15 src/XF.php(391): XF\App->run()
#16 index.php(16): XF::runApp('XF\\Api\\App')
#17 {main}

Request state
array(4) {
  ["url"] => string(75) "/api/dbtech-ecommerce/purchased?category_ids%5B0%5D=2&platforms%5B0%5D=xf21"
  ["referrer"] => bool(false)
  ["_GET"] => array(3) {
    ["/api/dbtech-ecommerce/purchased"] => string(0) ""
    ["category_ids"] => array(1) {
      [0] => string(1) "2"
    }
    ["platforms"] => array(1) {
      [0] => string(4) "xf21"
    }
  }
  ["_POST"] => array(0) {
  }
}

 

[Fillip H.]

This is because it's trying to encode the $_SERVER variable. Check that in XF's php info page to make sure there's nothing out of place.

 

[wmtech]

The $_SERVER['HTTP_USER_AGENT'] could be anything. But I cannot see any problems when looking at my values.

Thanks.

 

[wmtech]

You never know if someone accesses the api with strange environment variables.

I propose to catch any exceptions with logging to not let the whole api request fail because of a small logging problem. Or -as we have done it in our installation- filter malformed characters before logging.

I don't want to post the code we changed here, but I'll send it to you via DM, may be you want to think about including it with your next release. It solved the problem for us.

Thanks for your fantastic software!

 

[Fillip H.]

I'll review it at some point this week, thanks for your contribution