Issue with Java SSLSocket

I'm currently working on a client-server project.
I've decided I should use SSL for security reasons.

I tried converting all Socket and ServerSocket objects to their SSL variants, but to no avail.
When I use the SSLServerSocket and connect to it, socket output on the server-side freezes in the middle of outputstream.write(byte[]).

Here are the functions I use to create the SSL Sockets, ContextController.CONTEXT is the SSLContext and Constants.DEBUG is a fast way for me to toggle SSL:

public static ServerSocket server(int port, int backlog) throws IOException {
    return Constants.DEBUG ? new ServerSocket(port, backlog) : CONTEXT.getServerSocketFactory().createServerSocket(port, backlog);
}

public static Socket client(InetSocketAddress address) throws IOException {
    return Constants.DEBUG ? new Socket(address.getHostName(), address.getPort())
            : ContextController.CONTEXT.getSocketFactory().createSocket(address.getHostName(), address.getPort());
}

public static Socket client() throws IOException {
    return Constants.DEBUG ? new Socket() : ContextController.CONTEXT.getSocketFactory().createSocket();
}

When Constants.DEBUG = true (i.e. SSL is off) it works, but when SSL is on it freezes as stated above after sending some data indefinitely. How can I fix this?

EDIT:
Here is the source of ContextController.CONTEXT:
(note, I know I should use an actual TrustManager but first I want this to work)

    if (server) {// load the keystore
        try (FileInputStream fis = new FileInputStream(ks)) {
            CONTEXT = SSLContext.getInstance("SSL");

            // load the keystore from the file
            KeyStore keystore = KeyStore.getInstance(KeyStore.getDefaultType());
            keystore.load(fis, PASSWORD);

            // setup the KeyManagerFactory (used by the server)
            KeyManagerFactory kmf = KeyManagerFactory.getInstance(KeyManagerFactory.getDefaultAlgorithm());
            kmf.init(keystore, PASSWORD);

            CONTEXT.init(kmf.getKeyManagers(), null, null);
        }
        catch (Exception e) {
        }
    }
    else {
        CONTEXT = SSLContext.getInstance("SSL");
        CONTEXT.init(null, new TrustManager[] { new X509TrustManager() {
            @Override
            public void checkClientTrusted(X509Certificate[] chain, String authType) {
            }

            @Override
            public void checkServerTrusted(X509Certificate[] chain, String authType) {
            }

            @Override
            public X509Certificate[] getAcceptedIssuers() {
                return null;
            }
        } }, null);
    }

EDIT 2:
Here is a stacktrace of where it freezes, note it doesn't crash, it hangs:

Thread [ClientExecThread #0] (Suspended)    
owns: AppOutputStream  (id=54)  
owns: ByteArrayOutputStream  (id=55)    
SocketOutputStream.socketWrite0(FileDescriptor, byte[], int, int) line: not available [native method]   
SocketOutputStream.socketWrite(byte[], int, int) line: 111  
SocketOutputStream.write(byte[], int, int) line: 155    
OutputRecord.writeBuffer(OutputStream, byte[], int, int, int) line: 431 
OutputRecord.write(OutputStream, boolean, ByteArrayOutputStream) line: 417  
SSLSocketImpl.writeRecordInternal(OutputRecord, boolean) line: 876  
SSLSocketImpl.writeRecord(OutputRecord, boolean) line: 847  
AppOutputStream.write(byte[], int, int) line: 123   
ByteArrayOutputStream.writeTo(OutputStream) line: 167   
SocketStream.streamPacket(Packet) line: 181 
ClientThread.lambda$8(Group) line: 150  
1427646530.run(Object) line: not available  
ClientThread.execute() line: 444    
ClientThread$ClientExecThread.run() line: 261

I've placed a breakpoint the line after SocketStream.streamPacket(Packet) 181 and it never reaches it.

I really don't know why this happened and I don't need all aspects of SSL (only the encryption part, that why I didn't implement the TrustManager) so I decided to implement a Diffie Hellman instead.

This works as intended and combining the DH with AES I have encrypted traffic per my requirements.

Thank you everyone for you help!