This work presents the dynamic model of an unmanned aerial manipulator (UAM) obtained from the perspective of the end-effector. This perspective allows designing single-layer whole-body based controllers to directly perform trajectory tracking of the pose of the end-effector. A robust linear mixed H2/H-infinito controller with D-stability constraints is designed and implemented in a hardware-in-the-loop framework. Numerical experiments are conducted in a high fidelity simulator to evaluate the controller's performance when the system is subjected disturbances.