Java RMI To use SecurityManager or not to use SecurityManager

After significant searching and trial and error, I still can not determine if a Security Manager is necessary and if so, how to make it work.

Code to start server:

    Registry registry;
    try {
        System.setProperty("java.security.policyfile", "\\\\...\\security.policy");
        if (System.getSecurityManager() == null) {
            RMISecurityManager securityManager = new RMISecurityManager();
            System.setSecurityManager(securityManager);
        }
        registry = LocateRegistry.createRegistry(port);
        registry.bind(serviceName, remote);
        System.out.println("RMI registry created.");
    } catch (RemoteException e) {
        //error means registry already exists
        System.out.println("RMI registry already exists.");
    }
    System.setProperty("java.rmi.server.hostname", hostURL);
    Naming.rebind(hostURL, remote);

Error Message on Server:

D:\>java -jar Server.jar
RMI server starting up
RMI registry created.
Exception in thread "main" ...Exception: Unable to start the Remote Server Server[UnicastServerRef [liveRef: [...]]
        at ServiceProvider.start(ServiceProvider.java:64)
        at Server.main(Server.java:76)
Caused by: java.security.AccessControlException: access denied ("java.util.PropertyPermission" "java.rmi.server.hostname" "write")
        at java.security.AccessControlContext.checkPermission(Unknown Source)
        at java.security.AccessController.checkPermission(Unknown Source)
        at java.lang.SecurityManager.checkPermission(Unknown Source)
        at java.lang.System.setProperty(Unknown Source)
        at ServiceProvider.start(ServiceProvider.java:61)
        ... 1 more

Policy file:

grant {
   permission java.security.AllPermission;
 }; 

Thank you to anyone who can help.

I'll admit I'm not even sure I need the Security Manager, but when I didn't use it, I got the below. At least here the server started and I was able to connect to it from the client (the IP address provided was not used to get the connection, this was figured out internally). However, I get the below error message when I try to use one of the methods from the server, even just "printTime", which is supposed to just display the time on the server side.

Exception in thread "main" java.rmi.ConnectException: Connection refused to host: ...IP...; nested exception is: 
java.net.ConnectException: Connection timed out: connect

Thanks again.

Edit:

OK, I have removed the security manager, because all classes needed are inside my interface which is included in both my client and my server projects. @EJP, you are clearly extremely knowledgeable when it comes tom Java and RMI. I value your comments.

I continue to get the Connection timed out error. I realize this is common, but I have tried everything I can think of.
I added the property: java.rmi.server.codebase with all of the jar files in my lib folder.

A friend indicated to me that my problem is I have included POJOs (Plain Old Java Objects) which are too complicated for RMI to transmit. But all of my POJOs are serializable. Also, I was able to run the server on my localhost and access it using a client app without issue. This tells me that RMI can do the work I'm asking.

I don't get this Connection error until I run the server on a remote machine and then start my client. My client is able to connect, but then gets hung up at the printTime method.

Code to run client:

  final Registry registry = LocateRegistry.getRegistry(hostURL, port);
  final HXTTDBFInterface serverIX = (HXTTDBFInterface) registry.lookup(serviceName);

  //Start with simple Server Connection Test
  serverIX.printTime();  //CONNECTION TIMED OUT ERROR HERE
  String time = serverIX.getTime();
  System.out.println("Time From Server: " + time);

If anyone has any suggestions for things to try, I am all ears. Thanks in advance.

Further Clarification: I used the command netstat -a and it showed:

LocalAddress       ForeignAddress    State
0.0.0.0:1099       SERVERNAME:0      LISTENING
[::]:1099          SERVERNAME:0      LISTENING
XX.XX.XX.XX:53373  SERVERNAME:1099   ESTABLISHED

(Among many other lines)

Does this mean that it's actually listening?

You only need to install a security manager for RMI if the peer is using the codebase facility to supply classes to you dynamically. Otherwise it is strictly up to you whether you use one or not. I would avoid it unless you know it's required.