Looks like ssh-copy-id is a script that simplifies copying the keys to a remote authorized_keys file...you could do it manually (it's a one time exercise per client). Either by ssh'ing into the remote PC and adding the value, or by pulling the file over to your local machine, editing it and putting it back (use scp, for that).
Doesn't look like the script includes any provision to change the default ssh port.