长沙网站建设公司,房地产信息网上查询系统,中国软件这个公司怎么样,网站建设技术app下载简介
Hadoop KMS是基于Hadoop的KeyProvider API的加密密钥管理服务器#xff0c;它提供了使用REST API通过HTTP进行通信的客户端和服务器组件。
客户端是一个KeyProvider实现#xff0c;使用KMS HTTP REST API与KMS交互。
KMS及其客户端具有内置的安全性#xff0c;它们支…简介
Hadoop KMS是基于Hadoop的KeyProvider API的加密密钥管理服务器它提供了使用REST API通过HTTP进行通信的客户端和服务器组件。
客户端是一个KeyProvider实现使用KMS HTTP REST API与KMS交互。
KMS及其客户端具有内置的安全性它们支持HTTP SPNEGO Kerberos身份验证和HTTPS安全传输。
KMS是一个Java Jetty web应用程序。
KMS与Hadoop结合可以实现HDFS客户端透明的数据加密传输以及细粒度的权限控制。
本文使用Hadoop 3.3.1 为例进行KMS服务配置启动及hdfs文件加密传输示例。
安装部署Hadoop KMS
利用keytool生成秘钥
keytool -genkey -alias sandbox -keystore /root/kms.jks -dname CNlocalhost, OUlocalhost, Olocalhost, LSH, STSH, CCN -keypass 123456 -storepass 123456 -validity 180将秘钥存储密码放在hadoop配置目录下
cd ${HADOOP_HOME}/etc/hadoop
echo 123456 kms.keystore.password配置 kms server端配置 kms-site.xml
?xml version1.0 encodingUTF-8?
!--Licensed under the Apache License, Version 2.0 (the License);you may not use this file except in compliance with the License.You may obtain a copy of the License athttp://www.apache.org/licenses/LICENSE-2.0Unless required by applicable law or agreed to in writing, softwaredistributed under the License is distributed on an AS IS BASIS,WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.See the License for the specific language governing permissions andlimitations under the License.
--!-- Put site-specific property overrides in this file. --configurationpropertynamehadoop.kms.http.port/namevalue9600/value/propertypropertynamehadoop.kms.key.provider.uri/namevaluejceks://file/${user.home}/kms.keystore/value/propertypropertynamehadoop.security.keystore.java-keystore-provider.password-file/namevaluekms.keystore.password/value/property!-- KMS缓存 --propertynamehadoop.kms.cache.enable/namevaluetrue/value/propertypropertynamehadoop.kms.cache.timeout.ms/namevalue600000/value/propertypropertynamehadoop.kms.current.key.cache.timeout.ms/namevalue30000/value/propertypropertynamehadoop.security.kms.encrypted.key.cache.size/namevalue500/value/propertypropertynamehadoop.security.kms.encrypted.key.cache.low.watermark/namevalue0.3/value/propertypropertynamehadoop.security.kms.encrypted.key.cache.num.fill.threads/namevalue2/value/propertypropertynamehadoop.security.kms.encrypted.key.cache.expiry/namevalue43200000/value/property!-- KMS 聚集Audit 日志 --propertynamehadoop.kms.aggregation.delay.ms/namevalue10000/value/property!-- KMS 代理用户配置 --propertynamehadoop.kms.proxyuser.#USER#.users/namevalue*/value/propertypropertynamehadoop.kms.proxyuser.#USER#.groups/namevalue*/value/propertypropertynamehadoop.kms.proxyuser.#USER#.hosts/namevalue*/value/property!-- KMS Delegation Token 配置 --propertynamehadoop.kms.authentication.delegation-token.update-interval.sec/namevalue86400/valuedescriptionHow often the master key is rotated, in seconds. Default value 1 day./description/propertypropertynamehadoop.kms.authentication.delegation-token.max-lifetime.sec/namevalue604800/valuedescriptionMaximum lifetime of a delagation token, in seconds. Default value 7 days./description/propertypropertynamehadoop.kms.authentication.delegation-token.renew-interval.sec/namevalue86400/valuedescriptionRenewal interval of a delagation token, in seconds. Default value 1 day./description/propertypropertynamehadoop.kms.authentication.delegation-token.removal-scan-interval.sec/namevalue3600/valuedescriptionScan interval to remove expired delegation tokens./description/property
/configuration配置 client 端 kms配置在core-site.xml中添加
!-- kms --propertynamehadoop.security.key.provider.path/namevaluekms://httpbp1:9600/kms/value/propertypropertynamehadoop.security.kms.client.encrypted.key.cache.size/namevalue500/value/propertypropertynamehadoop.security.kms.client.encrypted.key.cache.low-watermark/namevalue0.3/value/propertypropertynamehadoop.security.kms.client.encrypted.key.cache.num.refill.threads/namevalue2/value/propertypropertynamehadoop.security.kms.client.encrypted.key.cache.expiry/namevalue43200000/value/property启动kms
cd ${HADOOP_HOME}
sbin/kms.sh start (遗弃)
sbin/kms.sh status (遗弃)hadoop --daemon start kmshadoop --daemon status kms重启hadoop
cd ${HADOOP_HOME}
sbin/stop-all.sh
sbin/start-all.sh测试kms
hadoop key create sandbox
hadoop key listhadoop fs -mkdir /aaaaa
hdfs crypto -createZone -keyName sandbox -path /aaaaa
hdfs crypto -listZones
