"Black box" Amazon: algorithm discriminate customers
noyb filed another complaint against Amazon Europe today. The e-commerce giant offers customers the possibility to pay for products later via "Monthly Invoice". A customer who was rejected for this payment method without any reasons given submitted an access request to Amazon in order to find out why he was rejected. The company refused to provide any information.
- Download: Complaint against Amazon (PDF)
- Download: English machine translation of the complaint (PDF)
The system decided fully automated. Just seconds after the order was confirmed, the customer received an automatic e-mail in which the payment by "Monthly Invoice" was rejected. Amazon urged him to switch to credit card payment, otherwise the order would be cancelled within 5 days. Neither a reason for the rejection was given, nor did customer service answer any questions in this regard.
GDPR requires transparency and verifiability. For automated individual decisions - such as whether or not to allow payment on account - a company must provide meaningful information about the logic involved and the scope of the underlying data processing. In Amazon's privacy policy, however, only vague information about any credit checking mechanisms are to be found.
Automated decisions must be verifiable by humans. This is obviously not possible with Amazon, as their billing department clarifies: "This automated decision can have various causes and cannot be adapted manually." Ironically, Amazon justifies this by saying that customer service cannot see the exact reason for the rejection "for data protection reasons". They also refused to clarify whether internal information or a negative credit score was used for the decision-making process involved.
Unsuccessful access request revealed further GDPR violations. Instead of being provided with a copy of the data, as required by Article 15(3) of the GDPR, the customer was supposed to manually download 54 folders with mostly incomprehensible tables. There was no information on the purposes or legal basis of the data processing provided. Also, Amazon refused to give out any information on data sources or recipients - although the GDPR requires controllers to provide such information. Enquiries from the customer were dismissed with insignificant text blocks.
"The extent to which Amazon ignores EU data protection law is concerning. Algorithms make decisions that not even their own employees can understand and check. Amazon's response to the access request also violates pretty much every paragraph of the applicable GDPR provisions." - Marco Blocher, data protection lawyer at noyb
Luxembourg authority largely inactive. As Amazon Europe is located in Luxembourg, the countries’ national data protection commission is responsible for dealing with the complaint. noyb has already filed a complaint against Amazon in January 2019; to date, no decision is in sight. Information on the status of the proceedings or access to files are refused since this would endanger the tasks of the authority and impair the purpose of the proceedings. We hope the complaint filed today will not disappear into the thicket of the Luxembourgian authority.
