I have an OpenSearch domain on Amazon OpenSearch Service. I use it for my asp.net core app. I now want to start using localhost by running the default docker-compose file from the quickstart docs, instead of using the Amazon OpenSearch Service (while in development). I run the docker-compose file and it is all good. But when I run my asp.net app I start to see this come through in the docker terminal:
"error","opensearch","data"],"pid":453,"message":"[ConnectionError]: connect ECONNREFUSED 172.18.0.4:9200"
EDIT: I also see this a lot:
Caused by: io.netty.handler.ssl.NotSslRecordException: not an SSL/TLS record
I'm pretty sure the only thing I should be changing in my asp.net app is these connection strings in appsettings.json:
"elasticEndpoint": "https://search-xxxxxxxxxxxxxxxxx.ap-southeast-2.es.amazonaws.com",
"elasticIndexName": "vepo",
"elasticUsername": "xxx",
"elasticPassword": "xxx"
So I changed it to:
"elasticEndpoint": "http://localhost:9200",
"elasticIndexName": "vepo",
"elasticUsername": "admin",
"elasticPassword": "admin"
As far as I know that should be all that is needed, anyone know if I'm missing something or did it wrong?
I will now dump a massive amount of info below that you probably don't need but I wrote it before I found the ECONNREFUSED error and it may be useful, who knows:
Every time I start the asp.net app it deletes the default index and recreates it, then recreates every other index and populates it from my Postgres database.
This is where i create the OpenSearch client:
public class SearchService : ConfigurableSearchService, ISearchService
{
private readonly string _defaultIndex;
private readonly bool _refreshOnUpdate;
private OpenSearchClient _client;
public static readonly Dictionary<Type, string> SearchIndexMappings = new Dictionary<Type, string>
{
{typeof(VgnItmSearchDto), "vgnitmsearchdto"},
{typeof(VgnItmEstSearchDto), "vgnitmestsearchdto"},
};
public string DefaultIndex { get => _defaultIndex; }
public SearchService(
string uri,
string username,
string password,
string defaultIndex,
bool forceInMemory = false,
bool refreshOnUpdate = false)
: base(SearchService.SearchIndexMappings)
{
ConnectionSettings connectionSettings;
_defaultIndex = defaultIndex;
_refreshOnUpdate = refreshOnUpdate;
var node = new Uri(uri);
IConnectionPool pool = new SingleNodeConnectionPool(node);
connectionSettings = forceInMemory
? new ConnectionSettings(pool, new InMemoryConnection())
: new ConnectionSettings(pool).BasicAuthentication(username, password);
connectionSettings = connectionSettings
.DefaultIndex(defaultIndex);
IConnectionSettingsValues settings = connectionSettings;
foreach (var mapping in SearchService.SearchIndexMappings)
{
settings.DefaultIndices.Add(mapping.Key, mapping.Value);
}
#if (DEBUG)
connectionSettings.DisableDirectStreaming();
#endif
_client = new OpenSearchClient(connectionSettings);
}
And that is called in Startup.cs, ConfigureServices():
services.AddScoped<ISearchService, SearchService>(serviceProvider =>
{
return new SearchService(
Configuration["elasticEndpoint"],
Configuration["elasticUsername"],
Configuration["elasticPassword"],
Configuration["elasticIndexName"]);
});
The Configuration is using this from appsettings.json, this was the AWS one:
"elasticEndpoint": "https://search-xxxxxxxxxxxxxxxxx.ap-southeast-2.es.amazonaws.com",
"elasticIndexName": "xxx",
"elasticUsername": "xxx",
"elasticPassword": "xxx"
So I changed it to:
"elasticEndpoint": "http://localhost:9200",
"elasticIndexName": "vepo",
"elasticUsername": "admin",
"elasticPassword": "admin"
At the bottom of Configure() I tell the searchService to reindex all:
searchIndexService.ReIndex(context, SearchIndexes.All);
To run OpenSearch locally, I have the default docker-compose file from the quick start docs:
version: '3'
services:
opensearch-node1:
image: opensearchproject/opensearch:latest
container_name: opensearch-node1
environment:
- cluster.name=opensearch-cluster
- node.name=opensearch-node1
- discovery.seed_hosts=opensearch-node1,opensearch-node2
- cluster.initial_cluster_manager_nodes=opensearch-node1,opensearch-node2
- bootstrap.memory_lock=true # along with the memlock settings below, disables swapping
- "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m" # minimum and maximum Java heap size, recommend setting both to 50% of system RAM
ulimits:
memlock:
soft: -1
hard: -1
nofile:
soft: 65536 # maximum number of open files for the OpenSearch user, set to at least 65536 on modern systems
hard: 65536
volumes:
- opensearch-data1:/usr/share/opensearch/data
ports:
- 9200:9200
- 9600:9600 # required for Performance Analyzer
networks:
- opensearch-net
opensearch-node2:
image: opensearchproject/opensearch:latest
container_name: opensearch-node2
environment:
- cluster.name=opensearch-cluster
- node.name=opensearch-node2
- discovery.seed_hosts=opensearch-node1,opensearch-node2
- cluster.initial_cluster_manager_nodes=opensearch-node1,opensearch-node2
- bootstrap.memory_lock=true
- "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m"
ulimits:
memlock:
soft: -1
hard: -1
nofile:
soft: 65536
hard: 65536
volumes:
- opensearch-data2:/usr/share/opensearch/data
networks:
- opensearch-net
opensearch-dashboards:
image: opensearchproject/opensearch-dashboards:latest
container_name: opensearch-dashboards
ports:
- 5601:5601
expose:
- "5601"
environment:
OPENSEARCH_HOSTS: '["https://opensearch-node1:9200","https://opensearch-node2:9200"]'
networks:
- opensearch-net
volumes:
opensearch-data1:
opensearch-data2:
networks:
opensearch-net:
I thought that if I run OpenSearch locally with docker-compose up, then I run my asp.net app, that it would then create my indexes in my local version of OpenSearch viewable at http://localhost:5601/app/home#/. But it doesn't. The Create index code all runs without any exceptions.
But my indexes do not get added there. And when I perform a search, in my asp.net app I get:
System.IO.IOException: The response ended prematurely. at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
And the OpenSearch docker terminal shows:
io.netty.handler.codec.DecoderException: io.netty.handler.ssl.NotSslRecordException: not an SSL/TLS record
What am I missing that seems to be not connecting my asp.net app to my localhost version of OpenSearch?
first: If you are running your app from a docker container not use localhost, use the local IP of the host machine.
I have read about the issue and I found that opensearch uses tls certificates for all connections: node-node, and client-node.
https://opensearch.org/docs/latest/security/configuration/tls/
So you could create and install the certificates by yourself to allow your app (client) to connect to the local cluster (node). But if you think is not necessary to install these certificates in the local deployment, I suggest you avoid the SSL verification.
before all, if you have curl (bash terminal) execute this command:
curl https://admin:admin@localhost:9200 -k
the -k asks curl not make ssl validation and admin:admin is username:password. And if the command works this solution will work for you.
in docker-composer.yaml add "DISABLE_SECURITY_PLUGIN=true" to the environments:
...
container_name: opensearch-node1
environment:
- cluster.name=opensearch-cluster # Name the cluster
- node.name=opensearch-node1 # Name the node that will run in this container
- discovery.seed_hosts=opensearch-node1,opensearch-node2 # Nodes to look for when discovering the cluster
- cluster.initial_cluster_manager_nodes=opensearch-node1,opensearch-node2 # Nodes eligible to serve as cluster manager
- bootstrap.memory_lock=true # Disable JVM heap memory swapping
- "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m" # Set min and max JVM heap sizes to at least 50% of system RAM
- "DISABLE_SECURITY_PLUGIN=true"
...
and
...
container_name: opensearch-node2
environment:
- cluster.name=opensearch-cluster
- node.name=opensearch-node2
- discovery.seed_hosts=opensearch-node1,opensearch-node2
- cluster.initial_cluster_manager_nodes=opensearch-node1,opensearch-node2
- bootstrap.memory_lock=true
- "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m"
- "DISABLE_SECURITY_PLUGIN=true"
...
now you can use http instead https:
curl http://admin:admin@localhost:9200
I'm not good at asp net core but I made a test disabling SSL validation and it worked(using HttpClient and your composer.yaml file).
var handler = new HttpClientHandler();
handler.ClientCertificateOptions = ClientCertificateOption.Manual;
handler.ServerCertificateCustomValidationCallback =
(httpRequestMessage, cert, cetChain, policyErrors) =>
{
return true;
};
string url = "https://localhost:9200";
HttpClient client = new HttpClient(handler);
client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Basic", "YWRtaW46YWRtaW4=");
using HttpResponseMessage response = await client.GetAsync(url);
var jsonResponse = await response.Content.ReadAsStringAsync();
Console.WriteLine($"{jsonResponse}\n");
YWRtaW46YWRtaW4= is a base64 string encode from "user:password" ("admin:admin")
the output:
{
"name" : "opensearch-node1",
"cluster_name" : "opensearch-cluster",
"cluster_uuid" : "mHWmnO2ORguqfe4F-0nJrQ",
"version" : {
"distribution" : "opensearch",
"number" : "2.8.0",
"build_type" : "tar",
"build_hash" : "db90a415ff2fd428b4f7b3f800a51dc229287cb4",
"build_date" : "2023-06-03T06:24:25.112415503Z",
"build_snapshot" : false,
"lucene_version" : "9.6.0",
"minimum_wire_compatibility_version" : "7.10.0",
"minimum_index_compatibility_version" : "7.0.0"
},
"tagline" : "The OpenSearch Project: https://opensearch.org/"
}
As you can see it works, so I think you should look into OpenSearch.Client package to know how to disable SSL and if it is possible, you have a solution.
Also, you can see the issue from outside of your application: your app needs an endpoint and doesn't need to concern about certificates. So we can provide an endpoint ready to consume without SSL certificates.
How ? Using an Nginx container as a reverse proxy for "https://opensearch-node1:9200". Nginx proxy by default does not validate SSL certificates, then you could make a request to http://nginx-service:80 and it will work.
steps:
server {
listen 80;
listen [::]:80;
location / {
proxy_pass https://opensearch-node1:9200;
}
}
nginx-proxy:
image: nginx:latest
container_name: nginx-proxy
ports:
- 80:80
volumes:
- ./nginx.conf:/etc/nginx/conf.d/default.conf:ro
networks:
- opensearch-net
docker-compose up
curl http://admin:admin@localhost
public class Student
{
public int Id { get; init; }
public string FirstName { get; init; }
public string LastName { get; init; }
public int GradYear { get; init; }
public double Gpa { get; init; }
}
string url = "http://localhost";
Console.WriteLine(url);
var node = new Uri(url);
var config = new ConnectionSettings(node).BasicAuthentication("admin", "admin").DefaultIndex("students");
var client = new OpenSearchClient(config);
Console.WriteLine($"{config}\n");
var student = new Student { Id = 100, FirstName = "Paulo", LastName = "Santos", Gpa = 3.93, GradYear = 2021 };
var response = client.Index(student, i => i.Index("students"));
Console.WriteLine($"{response}\n");
Valid OpenSearch.Client response built from a successful (201) low level call on PUT: /students/_doc/100
curl http://admin:admin@localhost/students/_doc/100
{"_index":"students","_id":"100","_version":2,"_seq_no":1,"_primary_term":2,"found":true,"_source":{"id":100,"firstName":"Paulo","lastName":"Santos","gradYear":2021,"gpa":3.93}}
I hope it helps you.