Elasticsearch教程,Elasticsearch安全篇,通过Nginx http basic 限制访问

soゝso 2017-02-10 11:24:06 7732

前言:

用  Elasticsearch  的同学都知道,最近一段时间  Elasticsearch  像中毒一样全国、全世界都发生的  Elasticsearch  被删库,有的甚至被加密后敲诈比特币。

当然我也难逃删库的命运,我200GB 的爬虫数据被删,我所在的公司只开放了公司IP 才能访问,也被删除了整个库。

那么问题来了,我们为什么要开放外网访问Elasticsearch?无外乎以下几点原因。

  1. 一些插件监听使用,方便及时了解线上数据的情况,比如说  head  插件等。
  2. 为了HTTP 直接访问,有的同学考虑到通过后台TCP 查询,然后返回数据,还不如直接以Elasticsearch 作为服务直接  HTTP  查询提升效率。
  3. 还有一些同学是因为Elasticsearch 和被访问的工程不在一个局域网内。
  4. 。。。。等

我的解决方案:

我是Centos Linux 系统,我直接用  iptables  限制  IP  访问,虽然不华丽,但是明显解决了。有个弊端就是在家里由于北方的宽带  IP  老变,经常要去加IP规则,比较痛苦。

但是我收到了阿里发的“【高危漏洞通告】  ElasticSearch  未授权访问漏洞”以下内容邮件:

尊敬的 ser****@sojson.com:
 
	您好,接上级主管部门通知,您的主机123.**.**.**安装有elasticsearch,目前elasticsearch有部分漏洞已被公布,存在信息泄露的隐患,请及时整改。如无法整改,经主管单位核实后,根据网络安全法的规定,会有关停主机的风险。
以下整改措施仅供参考:
一、监管部门加固方案
(一)elasticsearch自身安全设置
1、为elasticsearch增加登录验证,可以使用官方推荐的shield插件,该插件为收费插件,可试用30天,免费的可以使用elasticsearch-http-basic,searchguard插件。插件可以通过运行Biplugin install [github-name]/repo-name。同时需要注意增加验证后,请勿使用弱口令。
2、架设nginx反向代理服务器,并设置http basic认证来实现elasticsearch的登录认证。
3、默认开启的9200端口和使用的端口不对外公布,或架设内网环境。
4、elasticsearch 早期版本在“CVE中文漏洞信息库”网站上已有部分漏洞被披露,建议使用1.7.1以上版本或使用最新版本程序。
(二)大数据安全
1、设置HADOOP为基础的大数据信息系统,只允许或通过特定的IP进行访问,同时该IP地址进行安全设置(使用防病毒程序,设置复杂密码,安装最新漏洞补丁,使用应用程序防火墙等)
2、为NOSQL这类大数据数据库(如mongodb,redis)设置复杂密码
3、在大数据信息系统建设完毕后及时进行登记保护测评,并定期对该类大数据信息系统进行安全检查。
(三) 服务器安全设置
1、在服务器端安装系统防护软件,实现对操作系统加固和WEB业务系统及网站的防护监测,实现对信息系统的立体化监测和防护
2、对服务器中开启的服务(如数据库、FTP、web服务等)设置复杂密码,并定期更换,增加系统安全性
3、及时更新服务器漏洞补丁,防止漏洞被利用
4、在互联网出口设置防火墙访问策略,只允许特定需要对外访问的端口通过,其他异常访问全部阻断
5、对重要信息系统服务、数据信息、资产等进行黑白名单和权限访问设置。 

所以我就按照Email内容着手去做安全。

首先收费的shield 肯定是排除的,其次Elasticsearch-http-basic 也随之排除,因为没我对应的版本,详细对应版本请看下面表格:

Version Mapping:
Http Basic Plugin elasticsearch
v1.5.1(master) 1.5.1, 1.5.2, 1.6.0, 1.7.0
v1.5.0 1.5.0
v1.4.0 1.4.0
v1.3.0 1.3.0
v1.2.0 1.2.0
1.1.0 1.0.0
1.0.4 0.90.7

github地址:https://github.com/Asquera/elasticsearch-http-basic

最后选用方案是比较简单的,采用Nginx http-basic ,我是采用域名的方式访问,隐蔽了一层。

Nginx Http-basic 方案步骤实施

一、选用一个域名,并且配置转发。

upstream es.sojson.com{
         server 127.0.0.1:9981 weight=1 max_fails=2;
         server 127.0.0.1:9982 weight=1 max_fails=2;
         server 127.0.0.1:9983 weight=1 max_fails=2;
         server 127.0.0.1:9984 weight=1 max_fails=2;
         server 127.0.0.1:9985 weight=1 max_fails=2;
}
location ~* / {
		proxy_set_header Host $host;
		proxy_set_header X-Real-IP $remote_addr;
		proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

		proxy_set_header Upgrade $http_upgrade;
		proxy_set_header Connection "upgrade";

		if ($host ~* es\.sojson\.com) {
			proxy_pass  http://es.sojson.com;
		}
}

因为我五个点都加了插件,所以可以做个负载均衡。

二、配置帐号密码访问方式。

location ~* / {
		proxy_set_header Host $host;
		proxy_set_header X-Real-IP $remote_addr;
		proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

		proxy_set_header Upgrade $http_upgrade;
		proxy_set_header Connection "upgrade";

		#关键点,配置帐号密码
		auth_basic "login";#提示信息
		auth_basic_user_file /usr/local/nginx/conf/vhosts/password/es; #密码文件(注意最好别挑事,直接写绝对路径,别相对路径)
		autoindex on;

		if ($host ~* es\.sojson\.com) {
			proxy_pass  http://es.sojson.com;
		}
}

三、配置密码帐号文件。

上面第二点配置中有一个auth_basic_user_file  选项,这个选项就是配置的密码访问规则。密码是采用Crypt (all Unix servers)  方式密的,本站有生成工具:http://www.sojson.com/htpasswd.html 。如下图:

然后把生成的内容复制到上面配置的路径文件中/usr/local/nginx/conf/vhosts/password/es;

vi /usr/local/nginx/conf/vhosts/password/es

然后把生成的admin:PJdMvp0Utzclm 插入进去,保存之前看前面有没有丢了字母,我老会丢一个第一个字母,导致坑了一会。保存退出重启  Nginx  即可。

四、测试访问

访问之前配置好的域名es.sojson.com 。出现以下画面后输入配置好的帐号密码(明文)测试通过。

五、关闭外网访问。

elasticsearch$ vi config/elasticsearch.yml 

然后修改部分配置,只需要配置network.host : 127.0.0.1 即可

#network.publish_host: 0.0.0.0
#networt.bind_host: 0.0.0.0
#network.host: 0.0.0.0
#只要配置这个即可
network.host: 127.0.0.1
#network.publish_host: 127.0.0.1

重启  Elasticsearch  ,打完收工。想要更安全一点,可以再加上  iptables  ,然后再加上访问频率限制,防止暴力破解。


版权所属:SO JSON在线解析

原文地址:https://www.sojson.com/blog/213.html

转载时必须以链接形式注明原始出处及本声明。


如果本文对你有帮助,那么请你赞助我,让我更有激情的写下去,帮助更多的人。

相关文章
Elasticsearch教程Elasticsearch安全篇通过Nginx http basic 限制访问
Elasticsearch教程(五) elasticsearch Mapping的创建
Elasticsearch教程(六) elasticsearch Client创建
Elasticsearch教程Elasticsearch count 查询,Elasticsearch 查询是否存在
Elasticsearch教程Elasticsearch配置文件 — elasticsearch.yml
Elasticsearch教程Elasticsearch Java API创建Mapping,指定分词器
Elasticsearch 教程Elasticsearch 日期查询详解,Elasticsearch Date 查询Java API
Elasticsearch教程(二),IK分词器安装
Elasticsearch 教程Elasticsearch部署阿里云集群,支持外网请求方式
Elasticsearch教程(九) elasticsearch 查询数据 | 分页查询
最新文章
液厂JSON数据-来自云端数据 FreeMarker template error (DEBUG mode; use RETHROW in production!): The following has evaluated to null or missing: ==> it.readCount [in template "www/jsonText/x_left.ftl" at line 113, column 145] ---- Tip: It's the step after the last dot that caused this error, not those before it. ---- Tip: If the failing expression is known to be legally refer to something that's sometimes null or missing, either specify a default value like myOptionalVar!myDefault, or use <#if myOptionalVar??>when-present<#else>when-missing. (These only cover the last step of the expression; to cover the whole expression, use parenthesis: (myOptionalVar.foo)!myDefault, (myOptionalVar.foo)?? ---- ---- FTL stack trace ("~" means nesting-related): - Failed at: ${it.readCount?string(0)} [in template "www/jsonText/x_left.ftl" in macro "left" at line 113, column 143] - Reached through: @api target="docleftInfoTag" userId="... [in template "www/jsonText/x_left.ftl" in macro "left" at line 4, column 1] - Reached through: @n_left.left "${entity.userId?default... [in template "www/jsonText/x_details_4.ftl" at line 103, column 9] ---- Java stack trace (for programmers): ---- freemarker.core.InvalidReferenceException: [... Exception message was already printed; see it above ...] at freemarker.core.InvalidReferenceException.getInstance(InvalidReferenceException.java:131) at freemarker.core.UnexpectedTypeException.newDesciptionBuilder(UnexpectedTypeException.java:77) at freemarker.core.UnexpectedTypeException.(UnexpectedTypeException.java:40) at freemarker.core.BuiltInsForMultipleTypes$stringBI._eval(BuiltInsForMultipleTypes.java:617) at freemarker.core.Expression.eval(Expression.java:78) at freemarker.core.MethodCall._eval(MethodCall.java:55) at freemarker.core.Expression.eval(Expression.java:78) at freemarker.core.Expression.evalAndCoerceToString(Expression.java:82) at freemarker.core.DollarVariable.accept(DollarVariable.java:41) at freemarker.core.Environment.visit(Environment.java:324) at freemarker.core.MixedContent.accept(MixedContent.java:54) at freemarker.core.Environment.visitByHiddingParent(Environment.java:345) at freemarker.core.IteratorBlock$IterationContext.executeNestedBlockInner(IteratorBlock.java:268) at freemarker.core.IteratorBlock$IterationContext.executeNestedBlock(IteratorBlock.java:220) at freemarker.core.IteratorBlock$IterationContext.accept(IteratorBlock.java:194) at freemarker.core.Environment.visitIteratorBlock(Environment.java:572) at freemarker.core.IteratorBlock.acceptWithResult(IteratorBlock.java:78) at freemarker.core.IteratorBlock.accept(IteratorBlock.java:64) at freemarker.core.Environment.visit(Environment.java:324) at freemarker.core.MixedContent.accept(MixedContent.java:54) at freemarker.core.Environment.visitByHiddingParent(Environment.java:345) at freemarker.core.ConditionalBlock.accept(ConditionalBlock.java:48) at freemarker.core.Environment.visit(Environment.java:324) at freemarker.core.MixedContent.accept(MixedContent.java:54) at freemarker.core.Environment.visit(Environment.java:324) at freemarker.core.Environment$NestedElementTemplateDirectiveBody.render(Environment.java:2255) at com.sojson.core.tags.WYFTemplateModel.execute(WYFTemplateModel.java:48) at freemarker.core.Environment.visit(Environment.java:389) at freemarker.core.UnifiedCall.accept(UnifiedCall.java:104) at freemarker.core.Environment.visitByHiddingParent(Environment.java:345) at freemarker.core.Environment.visitAndTransform(Environment.java:425) at freemarker.core.UnifiedCall.accept(UnifiedCall.java:107) at freemarker.core.Environment.visit(Environment.java:324) at freemarker.core.Macro$Context.runMacro(Macro.java:184) at freemarker.core.Environment.invoke(Environment.java:701) at freemarker.core.UnifiedCall.accept(UnifiedCall.java:84) at freemarker.core.Environment.visit(Environment.java:324) at freemarker.core.MixedContent.accept(MixedContent.java:54) at freemarker.core.Environment.visit(Environment.java:324) at freemarker.core.Environment.process(Environment.java:302) at freemarker.template.Template.process(Template.java:325) at org.springframework.web.servlet.view.freemarker.FreeMarkerView.processTemplate(FreeMarkerView.java:367) at org.springframework.web.servlet.view.freemarker.FreeMarkerView.doRender(FreeMarkerView.java:284) at org.springframework.web.servlet.view.freemarker.FreeMarkerView.renderMergedTemplateModel(FreeMarkerView.java:234) at org.springframework.web.servlet.view.AbstractTemplateView.renderMergedOutputModel(AbstractTemplateView.java:167) at org.springframework.web.servlet.view.AbstractView.render(AbstractView.java:303) at org.springframework.web.servlet.DispatcherServlet.render(DispatcherServlet.java:1286) at org.springframework.web.servlet.DispatcherServlet.processDispatchResult(DispatcherServlet.java:1041) at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:984) at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:901) at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:970) at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:861) at javax.servlet.http.HttpServlet.service(HttpServlet.java:621) at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846) at javax.servlet.http.HttpServlet.service(HttpServlet.java:728) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210) at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:51) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210) at org.tuckey.web.filters.urlrewrite.RuleChain.handleRewrite(RuleChain.java:176) at org.tuckey.web.filters.urlrewrite.RuleChain.doRules(RuleChain.java:145) at org.tuckey.web.filters.urlrewrite.UrlRewriter.processRequest(UrlRewriter.java:92) at org.tuckey.web.filters.urlrewrite.UrlRewriteFilter.doFilter(UrlRewriteFilter.java:224) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210) at org.apache.catalina.core.ApplicationDispatcher.invoke(ApplicationDispatcher.java:749) at org.apache.catalina.core.ApplicationDispatcher.processRequest(ApplicationDispatcher.java:487) at org.apache.catalina.core.ApplicationDispatcher.doForward(ApplicationDispatcher.java:412) at org.apache.catalina.core.ApplicationDispatcher.forward(ApplicationDispatcher.java:339) at org.tuckey.web.filters.urlrewrite.NormalRewrittenUrl.doRewrite(NormalRewrittenUrl.java:213) at org.tuckey.web.filters.urlrewrite.RuleChain.handleRewrite(RuleChain.java:171) at org.tuckey.web.filters.urlrewrite.RuleChain.doRules(RuleChain.java:145) at org.tuckey.web.filters.urlrewrite.UrlRewriter.processRequest(UrlRewriter.java:92) at org.tuckey.web.filters.urlrewrite.UrlRewriteFilter.doFilter(UrlRewriteFilter.java:224) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210) at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:61) at org.apache.shiro.web.servlet.AdviceFilter.executeChain(AdviceFilter.java:108) at org.apache.shiro.web.servlet.AdviceFilter.doFilterInternal(AdviceFilter.java:137) at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125) at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:66) at org.apache.shiro.web.servlet.AbstractShiroFilter.executeChain(AbstractShiroFilter.java:449) at org.apache.shiro.web.servlet.AbstractShiroFilter$1.call(AbstractShiroFilter.java:365) at org.apache.shiro.subject.support.SubjectCallable.doCall(SubjectCallable.java:90) at org.apache.shiro.subject.support.SubjectCallable.call(SubjectCallable.java:83) at org.apache.shiro.subject.support.DelegatingSubject.execute(DelegatingSubject.java:383) at org.apache.shiro.web.servlet.AbstractShiroFilter.doFilterInternal(AbstractShiroFilter.java:362) at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:262) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210) at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:197) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:502) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:100) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408) at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1041) at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:603) at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:310) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745)