카프카 커넥트 실행 오류 [Failed to find any class that implements Connector and which name matches io.confluent.connect.jdbc.JdbcSourceConnector, available connectors are]

주키퍼와 카프카를 실행시키고

카프카 커넥트를 실행시켰는데

카프카 커넥트 실행 후 git bash 모습

이 상태에서

$ ./bin/windows/connect-distributed.bat ./etc/kafka/connect-distributed.properties
log4j:WARN No appenders could be found for logger (org.apache.kafka.connect.runtime.WorkerInfo).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
9▒▒ 01, 2022 1:15:35 ▒▒▒▒ org.glassfish.jersey.internal.inject.Providers checkProviderRuntime
▒▒▒: A provider org.apache.kafka.connect.runtime.rest.resources.ConnectorPluginsResource registered in SERVER runtime does not implement any provider interfaces applicable in the SERVER runtime. Due to constraint configuration problems the provider org.apache.kafka.connect.runtime.rest.resources.ConnectorPluginsResource will be ignored.
9▒▒ 01, 2022 1:15:35 ▒▒▒▒ org.glassfish.jersey.internal.inject.Providers checkProviderRuntime
▒▒▒: A provider org.apache.kafka.connect.runtime.rest.resources.RootResource registered in SERVER runtime does not implement any provider interfaces applicable in the SERVER runtime. Due to constraint configuration problems the provider org.apache.kafka.connect.runtime.rest.resources.RootResource will be ignored.
9▒▒ 01, 2022 1:15:35 ▒▒▒▒ org.glassfish.jersey.internal.inject.Providers checkProviderRuntime
▒▒▒: A provider org.apache.kafka.connect.runtime.rest.resources.ConnectorsResource registered in SERVER runtime does not implement any provider interfaces applicable in the SERVER runtime. Due to constraint configuration problems the provider org.apache.kafka.connect.runtime.rest.resources.ConnectorsResource will be ignored.
9▒▒ 01, 2022 1:15:35 ▒▒▒▒ org.glassfish.jersey.internal.inject.Providers checkProviderRuntime
▒▒▒: A provider org.apache.kafka.connect.runtime.rest.resources.LoggingResource registered in SERVER runtime does not implement any provider interfaces applicable in the SERVER runtime. Due to constraint configuration problems the provider org.apache.kafka.connect.runtime.rest.resources.LoggingResource will be ignored.
9▒▒ 01, 2022 1:15:35 ▒▒▒▒ org.glassfish.jersey.internal.Errors logErrors
▒▒▒: The following warnings have been detected: WARNING: The (sub)resource method listLoggers in org.apache.kafka.connect.runtime.rest.resources.LoggingResource contains empty path annotation.
WARNING: The (sub)resource method createConnector in org.apache.kafka.connect.runtime.rest.resources.ConnectorsResource contains empty path annotation.
WARNING: The (sub)resource method listConnectors in org.apache.kafka.connect.runtime.rest.resources.ConnectorsResource contains empty path annotation.
WARNING: The (sub)resource method listConnectorPlugins in org.apache.kafka.connect.runtime.rest.resources.ConnectorPluginsResource contains empty path annotation.
WARNING: The (sub)resource method serverInfo in org.apache.kafka.connect.runtime.rest.resources.RootResource contains empty path annotation.

 

127.0.0.1:8083/connectors 로 POST요청을 해보니까

BODY 내용은 JSON

{
    "name" : "my-source-connect",
    "config" : {
        "connector.class" : "io.confluent.connect.jdbc.JdbcSourceConnector",
        "connection.url":"jdbc:mysql://localhost:3307/mydb",
        "connection.user":"root",
        "connection.password":"여기엔 비밀번호 입력",
        "mode": "incrementing",
        "incrementing.column.name" : "id",
        "table.whitelist":"users",
        "topic.prefix" : "my_topic_",
        "tasks.max" : "1"
    }
}

다음과 같은 에러가 발생했다.

{
    "error_code": 500,
    "message": "Failed to find any class that implements Connector and which name matches io.confluent.connect.jdbc.JdbcSourceConnector, available connectors are: 
    PluginDesc{klass=class org.apache.kafka.connect.file.FileStreamSinkConnector, name='org.apache.kafka.connect.file.FileStreamSinkConnector', version='6.1.0-ccs', encodedVersion=6.1.0-ccs, type=sink, typeName='sink', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.file.FileStreamSourceConnector, name='org.apache.kafka.connect.file.FileStreamSourceConnector', version='6.1.0-ccs', encodedVersion=6.1.0-ccs, type=source, typeName='source', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.mirror.MirrorCheckpointConnector, name='org.apache.kafka.connect.mirror.MirrorCheckpointConnector', version='1', encodedVersion=1, type=source, typeName='source', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.mirror.MirrorHeartbeatConnector, name='org.apache.kafka.connect.mirror.MirrorHeartbeatConnector', version='1', encodedVersion=1, type=source, typeName='source', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.mirror.MirrorSourceConnector, name='org.apache.kafka.connect.mirror.MirrorSourceConnector', version='1', encodedVersion=1, type=source, typeName='source', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.tools.MockConnector, name='org.apache.kafka.connect.tools.MockConnector', version='6.1.0-ccs', encodedVersion=6.1.0-ccs, type=connector, typeName='connector', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.tools.MockSinkConnector, name='org.apache.kafka.connect.tools.MockSinkConnector', version='6.1.0-ccs', encodedVersion=6.1.0-ccs, type=sink, typeName='sink', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.tools.MockSourceConnector, name='org.apache.kafka.connect.tools.MockSourceConnector', version='6.1.0-ccs', encodedVersion=6.1.0-ccs, type=source, typeName='source', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.tools.SchemaSourceConnector, name='org.apache.kafka.connect.tools.SchemaSourceConnector', version='6.1.0-ccs', encodedVersion=6.1.0-ccs, type=source, typeName='source', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.tools.VerifiableSinkConnector, name='org.apache.kafka.connect.tools.VerifiableSinkConnector', version='6.1.0-ccs', encodedVersion=6.1.0-ccs, type=source, typeName='source', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.tools.VerifiableSourceConnector, name='org.apache.kafka.connect.tools.VerifiableSourceConnector', version='6.1.0-ccs', encodedVersion=6.1.0-ccs, type=source, typeName='source', location='classpath'}"
}

그래서 

Failed to find any class that implements Connector and which name matches io.confluent.connect.jdbc.JdbcSourceConnector, available connectors are

를 검색해 본결과

Kafka Connect에서 사용할 수 있는 Connectors plugin이 잘못 설치 된것 같았다.

 

해당 호출로인해 플러그인 목록을 확인해보면

http://localhost:8083/connector-plugins

JdbcSourceConnector 목록은 없었다.

 

즉, 이건 connect-distributed.properties 내용중

path.plugins 의 경로가 잘못됐다는 것이다.

자세히 봐보니 경로가 아래처럼 돼있었다. 윈도우에서는 \가 하나 더 붙어야한다.

윈도우에서는 아래처럼 \가 하나 더 붙어야한다.

 

다시

http://localhost:8083/connector-plugins을 요청해보니

{

• class: "io.confluent.connect.jdbc.JdbcSourceConnector",
• type: "source",
• version: "10.5.2"

}

정보가 있었다.

 

그리고 다시 127.0.0.1:8083/connectors 로 POST요청을 하니 정상 작동했다.