is needed, when you have it installed on a hidden path, but this is stored in your home directory, and will found 
Agreed, but: how do you know where it is actually stored ? Isn't that just an assumption ?
while i have it here generated
xmodmap -pke > Xmodmap-my
But yes, you can use, maybe
xmodmap $HOME/Xmodmap-my &When ever you will save it in your hidden config folder, you must set correct path, for example
xmodmap $HOME/.config/Xmodmap-my &