Charged a delivery fee for items not delivered

I really don't know whose bright idea is it to create a delivery app where you can't communicate with the driver. Driver came close to my place, did not follow instructions that were part of my order requesting that he press a number on my intercom for me to open. I sent a delivery note update with the question "is everything ok?" which was ignored. Then when I'm refunded, it's only for the items, not the delivery fee for items not delivered despite the fact that I made an effort to ask the driver whether everything is ok, I'm penalized and need to pay for delivery fee again in reordering. I understand hiring people who don't know how to follow rules of the road, but at least hire people who can at least read. Really goes against treating clients fairly to be penalized because someone decided not to follow instructions and press the intercom so that I can open for them.