Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
80 commits
Select commit Hold shift + click to select a range
1074d0f
[activemq] - change to ECS version git@v8.11.0
harnish-crest-data Jun 18, 2024
cb28736
[airflow] - change to ECS version git@v8.11.0
harnish-crest-data Jun 18, 2024
5019edf
[apache] - change to ECS version git@v8.11.0
harnish-crest-data Jun 18, 2024
b3d8c0d
[apache_spark] - change to ECS version git@v8.11.0
harnish-crest-data Jun 18, 2024
104c376
[apache_tomcat] - change to ECS version git@v8.11.0
harnish-crest-data Jun 18, 2024
7248d01
[awsfargate] - change to ECS version git@v8.11.0
harnish-crest-data Jun 18, 2024
bb6b01b
[azure_application_insights] - change to ECS version git@v8.11.0
harnish-crest-data Jun 18, 2024
ae3ab6a
[azure_app_service] - change to ECS version git@v8.11.0
harnish-crest-data Jun 18, 2024
841ec38
[azure_billing] - change to ECS version git@v8.11.0
harnish-crest-data Jun 18, 2024
21184f4
[azure_functions] - change to ECS version git@v8.11.0
harnish-crest-data Jun 18, 2024
17b51bb
[azure_openai] - change ecs.version to 8.11.0 in sample_event.json
harnish-crest-data Jun 18, 2024
0cc05bd
[ceph] - change to ECS version git@v8.11.0
harnish-crest-data Jun 18, 2024
0d72e30
[citrix_adc] - change to ECS version git@v8.11.0
harnish-crest-data Jun 18, 2024
c0f273f
[cockroachdb] - change to ECS version git@v8.11.0
harnish-crest-data Jun 18, 2024
0b61d9f
[coredns] - change to ECS version git@v8.11.0
harnish-crest-data Jun 18, 2024
f7af69d
[couchbase] - change to ECS version git@v8.11.0
harnish-crest-data Jun 18, 2024
446c8c3
[couchdb] - change to ECS version git@v8.11.0
harnish-crest-data Jun 18, 2024
2c8e3c0
[etcd] - change to ECS version git@v8.11.0
harnish-crest-data Jun 18, 2024
21dc7a0
[golang] - change to ECS version git@v8.11.0
harnish-crest-data Jun 18, 2024
45d9bde
[hadoop] - change to ECS version git@v8.11.0
harnish-crest-data Jun 18, 2024
967e797
[haproxy] - change to ECS version git@v8.11.0
harnish-crest-data Jun 18, 2024
849a17d
[ibmmq] - change to ECS version git@v8.11.0
harnish-crest-data Jun 18, 2024
cb20495
[iis] - change to ECS version git@v8.11.0
harnish-crest-data Jun 18, 2024
339892e
[influxdb] - change to ECS version git@v8.11.0
harnish-crest-data Jun 18, 2024
7c8c3ca
[jolokia] - change to ECS version git@v8.11.0
harnish-crest-data Jun 18, 2024
437c6d4
[kafka] - change to ECS version git@v8.11.0
harnish-crest-data Jun 18, 2024
671baf2
[kafka_log] - change to ECS version git@v8.11.0
harnish-crest-data Jun 18, 2024
a1b2e09
[memcached] - change to ECS version git@v8.11.0
harnish-crest-data Jun 18, 2024
a7f1d52
[microsoft_sqlserver] - change to ECS version git@v8.11.0
harnish-crest-data Jun 18, 2024
5417719
[mongodb] - change to ECS version git@v8.11.0
harnish-crest-data Jun 18, 2024
97ade70
[mysql] - change to ECS version git@v8.11.0
harnish-crest-data Jun 18, 2024
2b45bc1
[nagios_xi] - change to ECS version git@v8.11.0
harnish-crest-data Jun 18, 2024
5438003
[nginx] - change to ECS version git@v8.11.0
harnish-crest-data Jun 18, 2024
1f04b3d
[oracle] - change to ECS version git@v8.11.0
harnish-crest-data Jun 18, 2024
bc70fcc
[php_fpm] - change to ECS version git@v8.11.0
harnish-crest-data Jun 18, 2024
75fa530
[postgresql] - change to ECS version git@v8.11.0
harnish-crest-data Jun 18, 2024
3545f46
[prometheus_input] - change to ECS version git@v8.11.0
harnish-crest-data Jun 18, 2024
934f643
[redis] - change to ECS version git@v8.11.0
harnish-crest-data Jun 18, 2024
6c490c0
[redisenterprise] - change to ECS version git@v8.11.0
harnish-crest-data Jun 18, 2024
2d1f277
[system] - change to ECS version git@v8.11.0
harnish-crest-data Jun 18, 2024
68d4bf8
[tomcat] - change to ECS version git@v8.11.0
harnish-crest-data Jun 18, 2024
72982f4
[traefik] - removed ecs import_mappings
harnish-crest-data Jun 18, 2024
1ddec2b
[vsphere] - change to ECS version git@v8.11.0
harnish-crest-data Jun 18, 2024
93adb2d
[websphere_application_server] - change to ECS version git@v8.11.0
harnish-crest-data Jun 18, 2024
fa1f5b3
[zookeeper] - change to ECS version git@v8.11.0
harnish-crest-data Jun 18, 2024
ab80f0c
minor change in pr link
harnish-crest-data Jun 18, 2024
12638fb
[airflow] - generate system tests
harnish-crest-data Jun 18, 2024
4515f9b
[apache_spark] generate system tests
harnish-crest-data Jun 18, 2024
f9e07f2
[apache_tomcat] generate system tests
harnish-crest-data Jun 18, 2024
c23ba72
[azure_application_insights] - generate system tests
harnish-crest-data Jun 18, 2024
4c6d5ab
[azure_billing] - generate system tests
harnish-crest-data Jun 18, 2024
d563ac7
[cockroachdb] - generate system tests
harnish-crest-data Jun 18, 2024
7c69d3a
[couchbase] - generate system tests
harnish-crest-data Jun 18, 2024
6997f0b
[couchdb] - generate system tests
harnish-crest-data Jun 18, 2024
a751e21
[hadoop] - generate system tests
harnish-crest-data Jun 18, 2024
e935133
[haproxy] - generate system tests
harnish-crest-data Jun 18, 2024
e2644ae
[ibmmq] - generate system tests
harnish-crest-data Jun 18, 2024
db9528c
[influxdb] - generate system tests
harnish-crest-data Jun 18, 2024
d47b7ca
[memcached] - generate system tests
harnish-crest-data Jun 18, 2024
347037d
[mongodb] - generate system tests
harnish-crest-data Jun 18, 2024
9b5f518
[mysql] - generate system tests
harnish-crest-data Jun 18, 2024
4eafe0b
[nginx] - generate system tests
harnish-crest-data Jun 18, 2024
098e7ba
[oracle] - generate system tests
harnish-crest-data Jun 18, 2024
075bf4f
[postgresql] - generate system tests
harnish-crest-data Jun 18, 2024
0283c00
[vsphere] - generate system tests
harnish-crest-data Jun 18, 2024
3ac3f7e
[webshpere_application_server] - generate system tests
harnish-crest-data Jun 18, 2024
a84f04a
Merge branch 'main' of github.com:harnish-elastic/integrations into e…
harnish-crest-data Jun 20, 2024
bd35cdb
update readme
niraj-elastic Jun 24, 2024
6c7f31d
Merge branch 'main' of github.com:harnish-elastic/integrations into e…
harnish-crest-data Jun 24, 2024
4d557a1
generate pipeline tests for apache access data stream
harnish-crest-data Jun 25, 2024
ca01225
update readme
niraj-elastic Jun 25, 2024
ac731a6
Merge branch 'ecs_mapping_migration' of https://github.com/harnish-el…
niraj-elastic Jun 25, 2024
f637e29
elastic-package check
harnish-crest-data Jun 25, 2024
e2477ff
Merge branch 'main' of github.com:harnish-elastic/integrations into e…
harnish-crest-data Jun 26, 2024
b590918
address review comments
harnish-crest-data Jun 26, 2024
f02f223
address review comments
harnish-crest-data Jul 3, 2024
6396f10
Merge branch 'main' of github.com:harnish-elastic/integrations into e…
harnish-crest-data Jul 3, 2024
2a7dfbb
remove invalid ecs from ecs.yml
harnish-crest-data Jul 5, 2024
dd18efb
revert changes for hadoop package due to system tests failure
harnish-crest-data Jul 8, 2024
0efb542
Merge branch 'main' of github.com:harnish-elastic/integrations into e…
harnish-crest-data Jul 9, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
2 changes: 1 addition & 1 deletion packages/activemq/_dev/build/build.yml
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
dependencies:
ecs:
reference: git@v8.5.1
reference: "git@v8.11.0"
20 changes: 20 additions & 0 deletions packages/activemq/_dev/build/docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -96,6 +96,10 @@ These logs are System logs of ActiveMQ.

{{event "log"}}

**ECS Field Reference**

Please refer to the following [document](https://www.elastic.co/guide/en/ecs/current/ecs-field-reference.html) for detailed information on ECS fields.

{{fields "log"}}

### Audit Logs
Expand All @@ -104,6 +108,10 @@ In secured environments, it is required to log every user management action. Act

{{event "audit"}}

**ECS Field Reference**

Please refer to the following [document](https://www.elastic.co/guide/en/ecs/current/ecs-field-reference.html) for detailed information on ECS fields.

{{fields "audit"}}

## Metrics
Expand All @@ -114,6 +122,10 @@ ActiveMQ brokers serve as implementations of the Java Messaging Service (JMS), a

{{event "broker"}}

**ECS Field Reference**

Please refer to the following [document](https://www.elastic.co/guide/en/ecs/current/ecs-field-reference.html) for detailed information on ECS fields.

{{fields "broker"}}

### Queue Metrics
Expand All @@ -122,6 +134,10 @@ Queues are FIFO (first-in, first-out) pipelines of messages produced and consume

{{event "queue"}}

**ECS Field Reference**

Please refer to the following [document](https://www.elastic.co/guide/en/ecs/current/ecs-field-reference.html) for detailed information on ECS fields.

{{fields "queue"}}

### Topic Metrics
Expand All @@ -130,4 +146,8 @@ Topics are subscription-based message broadcast channels. When a producing appli

{{event "topic"}}

**ECS Field Reference**

Please refer to the following [document](https://www.elastic.co/guide/en/ecs/current/ecs-field-reference.html) for detailed information on ECS fields.

{{fields "topic"}}
5 changes: 5 additions & 0 deletions packages/activemq/changelog.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,9 @@
# newer versions go on top
- version: "1.4.0"
changes:
- description: ECS version updated to 8.11.0. Update the kibana constraint to ^8.13.0. Modified the field definitions to remove ECS fields made redundant by the ecs@mappings component template.
type: enhancement
link: https://github.com/elastic/integrations/pull/10162
- version: 1.3.0
changes:
- description: Add global filter on data_stream.dataset to improve performance.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,10 +8,10 @@
}
},
"ecs": {
"version": "8.5.1"
"version": "8.11.0"
},
"event": {
"ingested": "2023-10-11T20:53:28.142058918Z",
"ingested": "2024-06-18T05:48:44.477401346Z",
"kind": "event",
"module": "activemq",
"original": "INFO | anonymous called org.apache.activemq.broker.jmx.QueueView.retryMessages[] at 27-11-2019 08:45:57,213 | qtp443290224-47",
Expand All @@ -38,10 +38,10 @@
}
},
"ecs": {
"version": "8.5.1"
"version": "8.11.0"
},
"event": {
"ingested": "2023-10-11T20:53:28.142069543Z",
"ingested": "2024-06-18T05:48:44.477443111Z",
"kind": "event",
"module": "activemq",
"original": "INFO | admin called org.apache.activemq.broker.jmx.QueueView.retryMessages[] at 27-11-2019 08:45:57,229 | qtp443290224-45",
Expand All @@ -68,10 +68,10 @@
}
},
"ecs": {
"version": "8.5.1"
"version": "8.11.0"
},
"event": {
"ingested": "2023-10-11T20:53:28.142070835Z",
"ingested": "2024-06-18T05:48:44.477454071Z",
"kind": "event",
"module": "activemq",
"original": "WARN | admin requested /admin/createDestination.action [JMSDestination='test' JMSDestinationType='queue' secret='4eb0bc3e-9d7a-4256-844c-24f40fda98f1' ] from 127.0.0.1 | qtp12205619-39",
Expand All @@ -98,10 +98,10 @@
}
},
"ecs": {
"version": "8.5.1"
"version": "8.11.0"
},
"event": {
"ingested": "2023-10-11T20:53:28.142071877Z",
"ingested": "2024-06-18T05:48:44.477459356Z",
"kind": "event",
"module": "activemq",
"original": "INFO | guest requested /admin/purgeDestination.action [JMSDestination='test' JMSDestinationType='queue' secret='eff6a932-1b58-45da-a64a-1b30b246cfc9' ] from 127.0.0.1 | qtp12205619-36",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ processors:
ignore_failure: true
- set:
field: ecs.version
value: 8.5.1
value: 8.11.0
ignore_empty_value: true
ignore_failure: true
- script:
Expand Down
167 changes: 1 addition & 166 deletions packages/activemq/data_stream/audit/fields/agent.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,180 +5,15 @@
footnote: 'Examples: If Metricbeat is running on an EC2 host and fetches data from its host, the cloud info contains the data about this machine. If Metricbeat runs on a remote machine outside the cloud and fetches data from a service running in the cloud, the field contains cloud data from the machine the service is running on.'
type: group
fields:
- name: account.id
level: extended
type: keyword
ignore_above: 1024
description: 'The cloud account or organization id used to identify different entities in a multi-tenant environment.

Examples: AWS account id, Google Cloud ORG Id, or other unique identifier.'
example: 666777888999
- name: availability_zone
level: extended
type: keyword
ignore_above: 1024
description: Availability zone in which this host is running.
example: us-east-1c
- name: instance.id
level: extended
type: keyword
ignore_above: 1024
description: Instance ID of the host machine.
example: i-1234567890abcdef0
- name: instance.name
level: extended
type: keyword
ignore_above: 1024
description: Instance name of the host machine.
- name: machine.type
level: extended
type: keyword
ignore_above: 1024
description: Machine type of the host machine.
example: t2.medium
- name: provider
level: extended
type: keyword
ignore_above: 1024
description: Name of the cloud provider. Example values are aws, azure, gcp, or digitalocean.
example: aws
- name: region
level: extended
type: keyword
ignore_above: 1024
description: Region in which this host is running.
example: us-east-1
- name: project.id
type: keyword
description: Name of the project in Google Cloud.
- name: image.id
type: keyword
description: Image ID for the cloud instance.
- name: container
title: Container
group: 2
description: 'Container fields are used for meta information about the specific container that is the source of information.

These fields help correlate data based containers from any runtime.'
type: group
fields:
- name: id
level: core
type: keyword
ignore_above: 1024
description: Unique container id.
- name: image.name
level: extended
type: keyword
ignore_above: 1024
description: Name of the image the container was built on.
- name: labels
level: extended
type: object
object_type: keyword
description: Image labels.
- name: name
level: extended
type: keyword
ignore_above: 1024
description: Container name.
- name: host
title: Host
group: 2
description: 'A host is defined as a general computing instance.

ECS host.* fields should be populated with details about the host on which the event happened, or from which the measurement was taken. Host types include hardware, virtual machines, Docker containers, and Kubernetes nodes.'
description: 'A host is defined as a general computing instance. ECS host.* fields should be populated with details about the host on which the event happened, or from which the measurement was taken. Host types include hardware, virtual machines, Docker containers, and Kubernetes nodes.'
type: group
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need a greoup here ?
Since now its a single field ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, we need this hierarchy as there are two descriptions currently present. One related to cloud and other related to cloud.image.id

fields:
- name: architecture
level: core
type: keyword
ignore_above: 1024
description: Operating system architecture.
example: x86_64
- name: domain
level: extended
type: keyword
ignore_above: 1024
description: 'Name of the domain of which the host is a member.

For example, on Windows this could be the host''s Active Directory domain or NetBIOS domain name. For Linux this could be the domain of the host''s LDAP provider.'
example: CONTOSO
default_field: false
- name: hostname
level: core
type: keyword
ignore_above: 1024
description: 'Hostname of the host.

It normally contains what the `hostname` command returns on the host machine.'
- name: id
level: core
type: keyword
ignore_above: 1024
description: 'Unique host id.

As hostname is not always unique, use values that are meaningful in your environment.

Example: The current usage of `beat.name`.'
- name: ip
level: core
type: ip
description: Host ip addresses.
- name: mac
level: core
type: keyword
ignore_above: 1024
description: Host mac addresses.
- name: name
level: core
type: keyword
ignore_above: 1024
description: 'Name of the host.

It can contain what `hostname` returns on Unix systems, the fully qualified domain name, or a name specified by the user. The sender decides which value to use.'
- name: os.family
level: extended
type: keyword
ignore_above: 1024
description: OS family (such as redhat, debian, freebsd, windows).
example: debian
- name: os.kernel
level: extended
type: keyword
ignore_above: 1024
description: Operating system kernel version as a raw string.
example: 4.4.0-112-generic
- name: os.name
level: extended
type: keyword
ignore_above: 1024
multi_fields:
- name: text
type: text
norms: false
default_field: false
description: Operating system name, without the version.
example: Mac OS X
- name: os.platform
level: extended
type: keyword
ignore_above: 1024
description: Operating system platform (such centos, ubuntu, windows).
example: darwin
- name: os.version
level: extended
type: keyword
ignore_above: 1024
description: Operating system version as a raw string.
example: 10.14.1
- name: type
level: core
type: keyword
ignore_above: 1024
description: 'Type of host.

For Cloud providers this can be the machine type like `t2.medium`. If vm, this could be the container, for example, or other information meaningful in your environment.'
- name: containerized
type: boolean
description: If the host is a container.
Expand Down
20 changes: 0 additions & 20 deletions packages/activemq/data_stream/audit/fields/ecs.yml

This file was deleted.

2 changes: 1 addition & 1 deletion packages/activemq/data_stream/audit/sample_event.json
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@
"type": "logs"
},
"ecs": {
"version": "8.5.1"
"version": "8.11.0"
},
"elastic_agent": {
"id": "46343e0c-0d8c-464b-a216-cacf63027d6f",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@
}
},
"ecs": {
"version": "8.5.1"
"version": "8.11.0"
},
"event": {
"category": [
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ description: Pipeline for parsing ActiveMQ broker metrics.
processors:
- set:
field: ecs.version
value: 8.5.1
value: 8.11.0
- set:
field: event.category
value: [web]
Expand Down
24 changes: 0 additions & 24 deletions packages/activemq/data_stream/broker/fields/ecs.yml
Original file line number Diff line number Diff line change
@@ -1,28 +1,6 @@
- external: ecs
name: ecs.version
- external: ecs
name: error.message
- external: ecs
name: event.category
- external: ecs
name: event.dataset
- external: ecs
name: event.duration
- external: ecs
name: event.ingested
- external: ecs
name: event.kind
- external: ecs
name: event.module
- external: ecs
name: event.type
- external: ecs
name: service.address
dimension: true
- external: ecs
name: service.type
- external: ecs
name: tags
- external: ecs
name: agent.id
dimension: true
Expand All @@ -47,5 +25,3 @@
- external: ecs
name: host.name
dimension: true
- external: ecs
name: host.ip
Loading