Total Fruit Drop... every year

SHM starts with ...
I have a Luisa plum (dwarf), Moorpark apricot (dwarf), forget-the-name peach, and All-in-One almond all within several metres of each other in a square. All planted in winter 2018.

I'm having problems with total fruit drop, year after year, during the fruit set period and until fruit is about the size of 1/2 a thumb. 4 seasons in and I've got no fruit except for the peach.

The Luisa plum is probably the worst. This year it didn't produce much blossom either and dropped everything immediately. I expected better this year due to more rain last year.
The apricot has never produced either for the same reason but this year is looking more promising and every year is producing more blossom.
The almond, despite being almost 3m tall and looking healthy, has 1 almond on it.
By contrast, the peach has been laden from year 1 consistently delivering bumper crops of good quality fruit.
Also a nearby apple tree produces more apples than is sane for a tree its size (despite having no known pollinator).

Does anyone with stone fruit tree experience know the most likely causes?

The site is sloping with 30cm of good soil over clay. It was a lawn for... probably forever. A 40cm circle around the trees is mulched with a bit of manure mixed in.
At this point trees get sunlight from early morning until 3pm.

I have concerns about soil nutrient deficiency but adding complete fertiliser doesn't seem to be helping. Is there any particular deficiency that would significantly contribute to fruit drop? The peach tree is more likely to receive fertiliser due it its position.

Insufficient water is definitely a candidate but the peach and apple trees don't appear bothered by the water levels. I do water the trees but not in any scientific way.