[couchdb][server] Add CouchDB package with Server data stream by yug-rajani · Pull Request #3649 · elastic/integrations

yug-rajani · 2022-07-04T13:48:27Z

Enhancement

What does this PR do?

Generated the skeleton of CouchDB integration package.
Added 1 data stream ( Server )
Added data collection logic.
Added the ingest pipelines.
Mapped fields according to the ECS schema and added Fields metadata in the appropriate yml files.
Added system test cases.
Same OOTB dashboards with agent-based integration as metricbeat module for CouchDB

Checklist

I have reviewed tips for building integrations and this pull request is aligned with them.
I have verified that all data streams collect metrics or logs.
I have added an entry to my package's changelog.yml file.
If I'm introducing a new feature, I have modified the Kibana version constraint in my package's manifest.yml file to point to the latest Elastic stack release (e.g. ^7.13.0).

How to test this PR locally

Clone integrations repo.
Install elastic-package locally.
Start elastic stack using elastic-package.
Move to integrations/packages/couchdb directory.
Run the following command to run tests.

elastic-package test

elasticmachine · 2022-07-04T14:03:54Z

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Start Time: 2022-07-18T10:27:50.268+0000
Duration: 16 min 24 sec

Test stats 🧪

Test	Results
Failed	0
Passed	5
Skipped	0
Total	5

🤖 GitHub comments

To re-run your PR in the CI, just comment with:

/test : Re-trigger the build.

elasticmachine · 2022-07-04T14:04:03Z

🌐 Coverage report

Name	Metrics % (`covered/total`)	Diff
Packages	100.0% (`1/1`)	💚
Files	100.0% (`1/1`)	💚 3.049
Classes	100.0% (`1/1`)	💚 3.049
Methods	100.0% (`7/7`)	💚 10.947
Lines	94.175% (`194/206`)	👍 4.007
Conditionals	100.0% (`0/0`)	💚

kush-elastic

LGTM!

packages/couchdb/data_stream/server/fields/fields.yml

packages/couchdb/data_stream/server/_dev/test/pipeline/test-server-metrics.json-expected.json

lalit-satapathy · 2022-07-14T06:03:03Z

Hi @yug-elastic,

I am having some trouble running this integration.

My couchDB is up and credentials are fine.

curl http://localhost:5984/
{"couchdb":"Welcome","version":"3.2.2","git_sha":"d5b746b7c","uuid":"aca7dc34f6431a77d2ee28a2fc9d5e93","features":["access-ready","partitioned","pluggable-storage-engines","reshard","scheduler"],"vendor":{"name":"The Apache Software Foundation"}}

But I get following error. Is the url user-id/password passed properly? what could be going wrong below.

[elastic_agent.metricbeat][error] Error fetching data for metricset http.json: error making http request: Get "http://localhost:5984/_node/_local/_stats": dial tcp 127.0.0.1:5984: connect: connection refused.

yug-rajani · 2022-07-14T06:24:27Z

Hi @yug-elastic,

I am having some trouble running this integration.

My couchDB is up and credentials are fine.
curl http://localhost:5984/
{"couchdb":"Welcome","version":"3.2.2","git_sha":"d5b746b7c","uuid":"aca7dc34f6431a77d2ee28a2fc9d5e93","features":["access-ready","partitioned","pluggable-storage-engines","reshard","scheduler"],"vendor":{"name":"The Apache Software Foundation"}}
But I get following error. Is the url user-id/password passed properly? what could be going wrong below.

[elastic_agent.metricbeat][error] Error fetching data for metricset http.json: error making http request: Get "http://localhost:5984/_node/_local/_stats": dial tcp 127.0.0.1:5984: connect: connection refused.

Hey @lalit-satapathy,
Thanks for reaching out. Your CouchDB instance looks fine. I doubt that the machine on which your elastic stack is running, is unable to access your CouchDB instance. Could you help me answer the questions below to better understand the problem:

Is your ELK stack and CouchDB instance running on the same local/virtual machine? I doubt if they are on the same network.
I was able to bind the port and get a CouchDB instance up and running using:
docker run -e COUCHDB_USER=admin -e COUCHDB_PASSWORD=password -p 5984:5984 -d couchdb
What is the response of curl http://localhost:5984/ or curl http://localhost:5984/_node/_local/_stats from the machine your elastic stack is running on?
Could you please let me know your hosts parameter in the configuration of your integration?

lalit-satapathy · 2022-07-14T06:52:52Z

Hi @yug-elastic,
I am having some trouble running this integration.
My couchDB is up and credentials are fine.
curl http://localhost:5984/
{"couchdb":"Welcome","version":"3.2.2","git_sha":"d5b746b7c","uuid":"aca7dc34f6431a77d2ee28a2fc9d5e93","features":["access-ready","partitioned","pluggable-storage-engines","reshard","scheduler"],"vendor":{"name":"The Apache Software Foundation"}}
But I get following error. Is the url user-id/password passed properly? what could be going wrong below.
[elastic_agent.metricbeat][error] Error fetching data for metricset http.json: error making http request: Get "http://localhost:5984/_node/_local/_stats": dial tcp 127.0.0.1:5984: connect: connection refused.
Hey @lalit-satapathy, Thanks for reaching out. Your CouchDB instance looks fine. I doubt that the machine on which your elastic stack is running, is unable to access your CouchDB instance. Could you help me answer the questions below to better understand the problem:

Is your ELK stack and CouchDB instance running on the same local/virtual machine? I doubt if they are on the same network.
I was able to bind the port and get a CouchDB instance up and running using:
docker run -e COUCHDB_USER=admin -e COUCHDB_PASSWORD=password -p 5984:5984 -d couchdb

What is the response of curl http://localhost:5984/ or curl http://localhost:5984/_node/_local/_stats from the machine your elastic stack is running on?

Could you please let me know your hosts parameter in the configuration of your integration?

I have fixed my local setup and the integration run is going fine now. I will review the results/dashboards sometime later.

yug-rajani · 2022-07-19T11:28:27Z

Hey @lalit-satapathy, did you get a chance to review the results/dashboards for this PR? Could you please do it so that we could merge this integration as it contains just one data stream which is contained as a part of this PR?

lalit-satapathy · 2022-07-21T13:55:36Z

Verified the results/dashboards using docker couchDB.

LGTM.

yug-rajani and others added 2 commits July 4, 2022 19:09

Add foundation, server data stream, dashboard

2d2f13d

Update changelog.yml

4cd4606

yug-rajani self-assigned this Jul 4, 2022

yug-rajani added enhancement New feature or request Integration:couchdb CouchDB Team:Service-Integrations Label for the Observability Service Integrations team labels Jul 4, 2022

yug-rajani linked an issue Jul 4, 2022 that may be closed by this pull request

Create Couch DB package #3511

Closed

8 tasks

yug-rajani requested review from kush-elastic and lalit-satapathy July 4, 2022 13:50

kush-elastic approved these changes Jul 5, 2022

View reviewed changes

yug-rajani and others added 3 commits July 8, 2022 11:30

Update API doc reference in README

0ba16a2

Merge branch 'elastic:main' into package_couchdb_server

f4ac903

Add meta fields in fields.yml and update version to 8.3.0

8802a18

lalit-satapathy reviewed Jul 12, 2022

View reviewed changes

Resolve review comments

62d7112

yug-rajani requested a review from lalit-satapathy July 13, 2022 10:04

yug-rajani and others added 2 commits July 18, 2022 14:44

Update stream.yml.hbs

5b696b2

Merge branch 'elastic:main' into package_couchdb_server

2ad16ee

lalit-satapathy approved these changes Jul 21, 2022

View reviewed changes

yug-rajani merged commit a7b59f3 into elastic:main Jul 22, 2022

andrewkroh added the New Integration Issue or pull request for creating a new integration package. label Aug 13, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[couchdb][server] Add CouchDB package with Server data stream#3649

[couchdb][server] Add CouchDB package with Server data stream#3649
yug-rajani merged 8 commits intoelastic:mainfrom
yug-rajani:package_couchdb_server

yug-rajani commented Jul 4, 2022 •

edited

Loading

Uh oh!

elasticmachine commented Jul 4, 2022 •

edited

Loading

Build stats

Test stats 🧪

Uh oh!

elasticmachine commented Jul 4, 2022 •

edited

Loading

Uh oh!

kush-elastic left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

lalit-satapathy commented Jul 14, 2022

Uh oh!

yug-rajani commented Jul 14, 2022

Uh oh!

lalit-satapathy commented Jul 14, 2022

Uh oh!

yug-rajani commented Jul 19, 2022

Uh oh!

lalit-satapathy commented Jul 21, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Conversation

yug-rajani commented Jul 4, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

Checklist

How to test this PR locally

Uh oh!

elasticmachine commented Jul 4, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

💚 Build Succeeded

Build stats

Test stats 🧪

🤖 GitHub comments

Uh oh!

elasticmachine commented Jul 4, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🌐 Coverage report

Uh oh!

kush-elastic left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

lalit-satapathy commented Jul 14, 2022

Uh oh!

yug-rajani commented Jul 14, 2022

Uh oh!

lalit-satapathy commented Jul 14, 2022

Uh oh!

yug-rajani commented Jul 19, 2022

Uh oh!

lalit-satapathy commented Jul 21, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

yug-rajani commented Jul 4, 2022 •

edited

Loading

elasticmachine commented Jul 4, 2022 •

edited

Loading

elasticmachine commented Jul 4, 2022 •

edited

Loading