by definition of surprise, you are surprised when you don't know something is going to happen, if you are attacked by a hidden attacker, or if an allie turns on you, or if you just have no way of knowing somthinig can happen, you are suprised
Thats your definition of surprise, not the games definition of surprise.
If an ally just launches an attack at you and you are aware of his presence beforehand, (he's standing in front of you) all that happens is you roll initiative. If he wins, he attacks you before you can act (however you still have enough time to initiate a split second reaction like the parry manouver, or the shield spell). If you win initiative then you beat him to the draw and can attack (or whatever you want to do with your action first). You see him draw his weapon, advance towards you, and attack and have just enough time to run away before he closes the distance, cast a spell or draw your weapon and attack him before his attack is resolved.
Thats the rules.
Its the same deal if a hidden creature starts a combat by declaring an attack. The declaration starts the combat round sequence. The attack is now in motion. Initiative is then rolled. The attack does not get resolved until that hidden persons initiative result comes up in the initiative order. If the opposing side notice just one hostile enemy prior to rolling initiative, they are prepared and ready for combat and not surprised on turn one; meaning they get a chance to act before the hidden creature. If they are surprised then even winning initiative wont help them.
Youre granting a whole free round of actions to the person that screams 'I ATTACK' the loudest and quickest. The conseqence of a free round is that a 'surprising foe' gets up to three full rounds of action before the other side can act if he rolls well on initiative. A 'free' round to shoot a volley at his enemies outside of the combat sequence, his first turn during the combat at his surprised foes, and then his second round of actions. Three full attacks before they can react is a death sentence in 5E.
In the RAW, its 2 max.
I wasn't clear enough, in the Skulker case, if the shot missed and the target is blind/deaf, there is no reason that the target knows that he is in combat.
And in extreme outliers like that (hidden assassin with the skulker feat, shoots arrow and blind and deaf target, and misses) then a DM would be within his rights to rule differently.
I would probably allow a blind and deaf PC a chance (perception check) to still notice the arrow wizz past his nose though. This doesnt reveal the location of the assassin (thanks to Skulker) it just puts the PC on notice that he's being attacked.
Next example: the wizard is at home, deeply engrossed in some grimoire. An assassin attempts to sneak up from behind, and shoot an arrow, trying to, y'know, assassinate the wizard.
The Stealth/Perception contest results in a win for the assassin; he remains undetected by the wizard.
As soon as the assassin declares that he is shooting the wizard, initiative is rolled. This models the reaction speed of all the participants. It does not model whether or not the wizard perceives the assassin, the arrow or that combat has begun; he has not noticed a threat, and his reaction speed doesn't come into it.
The wizard rolls a higher initiative. His reaction will be faster than the assassin, but the wizard still hasn't noticed anything to react to.
Before the wizard's initiative, he cannot use reactions.
During his initiative, he cannot move or act because he is surprised.
After his initiative but before the assassin's, the wizard is able to react, but there is nothing to react to.
YES THERE IS.
The assassin has already declared his action and commenced it (however it has yet to be resolved). This is what has started the combat.
If you're the DM, you narrate this encounter as "You hear the creak of a bow from behind you in the darkness, and see an arrow fly towards you. Roll initiative.'
Not: You see an arrow sticking out of your chest. Roll initiat... oh wait, youre dead.'
On the assassin's initiative, he shoots an invisible silenced arrow from hidden, but he misses. The Skulker feat means that he remains hidden, and the invisible, silenced arrow arcs through a window into the night, disturbing nothing.
See above. How do we know the Wizard noticed nothing? How do we know it flew through a window?
Although like I said above, as an extreme outlier example, I might just inform the Wizards player that he notices nothing and is still unaware combat has commenced.
He might get a bit suss that I've already made him roll initiative of course, but a mature player doesnt metagame (or the next arrow might mysteriously hit automatically).
Casting shield is a reaction, and it represents throwing up a shield at the last moment, turning what would have been a hit into a miss instead.
You cant cast shield against an invisible silenced arrow fired at you from a hidden foe can you? I certainly wouldnt allow it.
Even though the wizard can use reactions, this does not give him the ability to react to things he doesn't know about. He hasn't noticed the arrow, so he can't put up a shield in response to an un-perceived trigger.
Exactly. Thats the rule with reactions generally though (and not the rules for surprise) - reactions require a trigger, and if you dont percieve the trigger, you cant use the reaction. A deflect arrows monk would be in the same troubles against a silent/ invisible arrow from a hidden assassin also.
Incidentaly another advantage of being hidden (or invisible or both) is that most spells cant be cast at you. Its hidden in the text of the individual spells.