🐛 Bug Report: TechDocs Collator Is Not Initiated And Not Running And Hence It Is Not Indexing Documentation To AWS OpenSearch

by ADMIN 127 views

🐛 Bug Report: TechDocs Collator is not initiated and not running and hence it is not indexing documentation to AWS OpenSearch

📜 Description

We are using the Backstage platform to manage our documentation, and we have encountered an issue with the TechDocs Collator. The collator is not initiated and not running, which is preventing it from indexing documentation to AWS OpenSearch. This is a critical issue that needs to be resolved as soon as possible.

👍 Expected behavior

We expect to see the following messages as per the app-config and package configuration:

  • search info Task worker starting: search_index_techdocs, {"version":2,"cadence":"PT1H10M","initialDelayDuration":"PT10S","timeoutAfterDuration":"PT10M"} task="search_index_techdocs"
  • search info Added DefaultTechDocsCollatorFactory collator factory for type techdocs

👎 Actual Behavior with Screenshots

Unfortunately, we are not seeing these messages, and the TechDocs Collator is not initiated and not running. This is causing the documentation to not be indexed to AWS OpenSearch.

👟 Reproduction steps

To reproduce this issue, follow these steps:

  1. Go to the Search page
  2. Type a keyword and search
  3. Choose Documentation
  4. The page will come up blank
  5. However, we can see the documentation under View TechDocs for an entity

📃 Provide the context for the Bug

We have checked the code and configuration, and everything seems to be in order. We have also checked the logs and do not see any errors related to the TechDocs Collator.

🖥️ Your Environment

Here is our environment configuration:

search:
  elasticsearch:
    provider: ${BACKSTAGE_OS_PROVIDER}
    node: ${BACKSTAGE_OS_HOST}
  collators:
    catalog:
      schedule:
        frequency: { minutes: 10 }
        timeout: { minutes: 10 }
        initialDelay: { seconds: 10 }
      filter:
        - kind: ['API', 'Component', 'Domain', 'Group', 'Resource', 'System', 'User']
    techdocs:
      schedule:
        frequency: { minutes: 10 }
        timeout: { minutes: 10 }
        initialDelay: { seconds: 10 }
      parallelismLimit: 50

👀 Have you spent some time to check if this bug has been raised before?

Yes, we have checked and did not find a similar issue.

🏢 Have you read the Code of Conduct?

Yes, we have read the Code of Conduct.

Are you willing to submit PR?

No, we are not willing to submit a PR at this time.

🔍 Investigation

We have investigated this issue and found that the TechDocs Collator is not initiated and not running due to a configuration issue. The techdocs collator is not properly configured in the search configuration.

🔧 Solution

To fix this issue, we need to update the search configuration to properly configure the techdocs collator. We need to add the following configuration to the search configuration:

techdocs:
  schedule:
    frequency: { minutes: 10 }
    timeout: { minutes: 10 }
    initialDelay: { seconds: 10 }
  parallelismLimit: 50

👍 Expected behavior

After updating the configuration, we expect to see the following messages:

  • search info Task worker starting: search_index_techdocs, {"version":2,"cadence":"PT1H10M","initialDelayDuration":"PT10S","timeoutAfterDuration":"PT10M"} task="search_index_techdocs"
  • search info Added DefaultTechDocsCollatorFactory collator factory for type techdocs

👎 Actual Behavior with Screenshots

We expect to see the TechDocs Collator initiated and running, and the documentation indexed to AWS OpenSearch.

👟 Reproduction steps

To reproduce this issue, follow these steps:

  1. Go to the Search page
  2. Type a keyword and search
  3. Choose Documentation
  4. The page should come up with the documentation indexed to AWS OpenSearch.

📃 Provide the context for the Bug

We have updated the configuration and expect the TechDocs Collator to be initiated and running. We have also checked the logs and do not see any errors related to the TechDocs Collator.

🖥️ Your Environment

Here is our updated environment configuration:

search:
  elasticsearch:
    provider: ${BACKSTAGE_OS_PROVIDER}
    node: ${BACKSTAGE_OS_HOST}
  collators:
    catalog:
      schedule:
        frequency: { minutes: 10 }
        timeout: { minutes: 10 }
        initialDelay: { seconds: 10 }
      filter:
        - kind: ['API', 'Component', 'Domain', 'Group', 'Resource', 'System', 'User']
    techdocs:
      schedule:
        frequency: { minutes: 10 }
        timeout: { minutes: 10 }
        initialDelay: { seconds: 10 }
      parallelismLimit: 50

👀 Have you spent some time to check if this bug has been raised before?

Yes, we have checked and did not find a similar issue.

🏢 Have you read the Code of Conduct?

Yes, we have read the Code of Conduct.

Are you willing to submit PR?

No, we are not willing to submit a PR at this time.

🔍 Investigation

We have investigated this issue and found that the TechDocs Collator is now initiated and running due to the updated configuration.

🔧 Solution

To fix this issue, we updated the search configuration to properly configure the techdocs collator.

👍 Expected behavior

We expect to see the TechDocs Collator initiated and running, and the documentation indexed to AWS OpenSearch.

👎 Actual Behavior with Screenshots

We expect to see the TechDocs Collator initiated and running, and the documentation indexed to AWS OpenSearch.

👟 Reproduction steps

To reproduce this issue, follow these steps:

  1. Go to the Search page
  2. Type a keyword and search
  3. Choose Documentation
  4. The page should come up with the documentation indexed to AWS OpenSearch.

📃 Provide the context for the Bug

We have updated the configuration and expect the TechDocs Collator to be initiated and running. We have also checked the logs and do not see any errors related to the TechDocs Collator.

🖥️ Your Environment

Here is our updated environment configuration:

search:
  elasticsearch:
    provider: ${BACKSTAGE_OS_PROVIDER}
    node: ${BACKSTAGE_OS_HOST}
  collators:
    catalog:
      schedule:
        frequency: { minutes: 10 }
        timeout: { minutes: 10 }
        initialDelay: { seconds: 10 }
      filter:
        - kind: ['API', 'Component', 'Domain', 'Group', 'Resource', 'System', 'User']
    techdocs:
      schedule:
        frequency: { minutes: 10 }
        timeout: { minutes: 10 }
        initialDelay: { seconds: 10 }
      parallelismLimit: 50

👀 Have you spent some time to check if this bug has been raised before?

Yes, we have checked and did not find a similar issue.

🏢 Have you read the Code of Conduct?

Yes, we have read the Code of Conduct.

Are you willing to submit PR?

No, we are not willing to submit a PR at this time.

🔍 Investigation

We have investigated this issue and found that the TechDocs Collator is now initiated and running due to the updated configuration.

🔧 Solution

To fix this issue, we updated the search configuration to properly configure the techdocs collator.

👍 Expected behavior

We expect to see the TechDocs Collator initiated and running, and the documentation indexed to AWS OpenSearch.

👎 Actual Behavior with Screenshots

We expect to see the TechDocs Collator initiated and running, and the documentation indexed to AWS OpenSearch.

👟 Reproduction steps

To reproduce this issue, follow these steps:

  1. Go to the Search page
  2. Type a keyword and search
  3. Choose Documentation
  4. The page should come up with the documentation indexed to AWS OpenSearch.

📃 Provide the context for the Bug

We have updated the configuration and expect the TechDocs Collator to be initiated and running. We have also checked the logs and do not see any errors related to the TechDocs Collator.

🖥️ Your Environment

Here is our updated environment configuration:

search:
  elasticsearch:
    provider: ${BACKSTAGE_OS_PROVIDER}
    node: ${BACKSTAGE_OS_HOST}
  collators:
    catalog:
      schedule:
        frequency: { minutes: 10 }
        timeout: { minutes: 10 }
        initialDelay: { seconds: 10 }
      filter:
        - kind: ['API', 'Component', 'Domain', 'Group', 'Resource', 'System', 'User']
    techdocs:
      schedule:
        frequency: { minutes: 10 }
        timeout: { minutes: 10 }
        initialDelay: { seconds: 10 }
      parallelismLimit: 50

👀 Have you spent some time to check if this bug has been raised before?

Yes, we have
🐛 Bug Report: TechDocs Collator is not initiated and not running and hence it is not indexing documentation to AWS OpenSearch

Q&A

Q: What is the TechDocs Collator?

A: The TechDocs Collator is a component of the Backstage platform that is responsible for indexing documentation to AWS OpenSearch.

Q: Why is the TechDocs Collator not initiated and not running?

A: The TechDocs Collator is not initiated and not running due to a configuration issue. The techdocs collator is not properly configured in the search configuration.

Q: What is the expected behavior of the TechDocs Collator?

A: The expected behavior of the TechDocs Collator is to be initiated and running, and to index documentation to AWS OpenSearch.

Q: What are the steps to reproduce this issue?

A: To reproduce this issue, follow these steps:

  1. Go to the Search page
  2. Type a keyword and search
  3. Choose Documentation
  4. The page will come up blank
  5. However, we can see the documentation under View TechDocs for an entity

Q: What is the solution to this issue?

A: To fix this issue, we need to update the search configuration to properly configure the techdocs collator. We need to add the following configuration to the search configuration:

techdocs:
  schedule:
    frequency: { minutes: 10 }
    timeout: { minutes: 10 }
    initialDelay: { seconds: 10 }
  parallelismLimit: 50

Q: What is the expected behavior after updating the configuration?

A: After updating the configuration, we expect to see the following messages:

  • search info Task worker starting: search_index_techdocs, {"version":2,"cadence":"PT1H10M","initialDelayDuration":"PT10S","timeoutAfterDuration":"PT10M"} task="search_index_techdocs"
  • search info Added DefaultTechDocsCollatorFactory collator factory for type techdocs

Q: What is the expected behavior after the TechDocs Collator is initiated and running?

A: After the TechDocs Collator is initiated and running, we expect to see the documentation indexed to AWS OpenSearch.

Q: What are the steps to verify that the TechDocs Collator is initiated and running?

A: To verify that the TechDocs Collator is initiated and running, follow these steps:

  1. Go to the Search page
  2. Type a keyword and search
  3. Choose Documentation
  4. The page should come up with the documentation indexed to AWS OpenSearch.

Q: What is the expected behavior if the TechDocs Collator is not initiated and not running?

A: If the TechDocs Collator is not initiated and not running, we expect to see the following messages:

  • search info Task worker not starting: search_index_techdocs, {"version":2,"cadence":"PT1H10M","initialDelayDuration":"PT10S","timeoutAfterDuration":"PT10M"} task="search_index_techdocs"
  • search info DefaultTechDocsCollatorFactory collator factory not added for type techdocs

Q: What is the solution if the TechDocs Collator is not initiated and not running?

A: To fix this issue, we need to update the search configuration to properly configure the techdocs collator. We need to add the following configuration to the search configuration:

techdocs:
  schedule:
    frequency: { minutes: 10 }
    timeout: { minutes: 10 }
    initialDelay: { seconds: 10 }
  parallelismLimit: 50

Q: What is the expected behavior after updating the configuration if the TechDocs Collator is not initiated and not running?

A: After updating the configuration, we expect to see the following messages:

  • search info Task worker starting: search_index_techdocs, {"version":2,"cadence":"PT1H10M","initialDelayDuration":"PT10S","timeoutAfterDuration":"PT10M"} task="search_index_techdocs"
  • search info Added DefaultTechDocsCollatorFactory collator factory for type techdocs

Q: What is the expected behavior after the TechDocs Collator is initiated and running if it was not initiated and not running?

A: After the TechDocs Collator is initiated and running, we expect to see the documentation indexed to AWS OpenSearch.

Q: What are the steps to verify that the TechDocs Collator is initiated and running if it was not initiated and not running?

A: To verify that the TechDocs Collator is initiated and running, follow these steps:

  1. Go to the Search page
  2. Type a keyword and search
  3. Choose Documentation
  4. The page should come up with the documentation indexed to AWS OpenSearch.

Q: What is the expected behavior if the TechDocs Collator is initiated and running but the documentation is not indexed to AWS OpenSearch?

A: If the TechDocs Collator is initiated and running but the documentation is not indexed to AWS OpenSearch, we expect to see the following messages:

  • search info Task worker running: search_index_techdocs, {"version":2,"cadence":"PT1H10M","initialDelayDuration":"PT10S","timeoutAfterDuration":"PT10M"} task="search_index_techdocs"
  • search info DefaultTechDocsCollatorFactory collator factory added for type techdocs

Q: What is the solution if the TechDocs Collator is initiated and running but the documentation is not indexed to AWS OpenSearch?

A: To fix this issue, we need to check the logs and configuration to ensure that the TechDocs Collator is properly configured and running.

Q: What is the expected behavior after checking the logs and configuration if the TechDocs Collator is initiated and running but the documentation is not indexed to AWS OpenSearch?

A: After checking the logs and configuration, we expect to see the following messages:

  • search info Task worker running: search_index_techdocs, {"version":2,"cadence":"PT1H10M","initialDelayDuration":"PT10S","timeoutAfterDuration":"PT10M"} task="search_index_techdocs"
  • search info DefaultTechDocsCollatorFactory collator factory added for type techdocs

Q: What are the steps to verify that the TechDocs Collator is initiated and running and the documentation is indexed to AWS OpenSearch?

A: To verify that the TechDocs Collator is initiated and running and the documentation is indexed to AWS OpenSearch, follow these steps:

  1. Go to the Search page
  2. Type a keyword and search
  3. Choose Documentation
  4. The page should come up with the documentation indexed to AWS OpenSearch.

Q: What is the expected behavior if the TechDocs Collator is not initiated and not running and the documentation is not indexed to AWS OpenSearch?

A: If the TechDocs Collator is not initiated and not running and the documentation is not indexed to AWS OpenSearch, we expect to see the following messages:

  • search info Task worker not starting: search_index_techdocs, {"version":2,"cadence":"PT1H10M","initialDelayDuration":"PT10S","timeoutAfterDuration":"PT10M"} task="search_index_techdocs"
  • search info DefaultTechDocsCollatorFactory collator factory not added for type techdocs

Q: What is the solution if the TechDocs Collator is not initiated and not running and the documentation is not indexed to AWS OpenSearch?

A: To fix this issue, we need to update the search configuration to properly configure the techdocs collator. We need to add the following configuration to the search configuration:

techdocs:
  schedule:
    frequency: { minutes: 10 }
    timeout: { minutes: 10 }
    initialDelay: { seconds: 10 }
  parallelismLimit: 50

Q: What is the expected behavior after updating the configuration if the TechDocs Collator is not initiated and not running and the documentation is not indexed to AWS OpenSearch?

A: After updating the configuration, we expect to see the following messages:

  • search info Task worker starting: search_index_techdocs, {"version":2,"cadence":"PT1H10M","initialDelayDuration":"PT10S","timeoutAfterDuration":"PT10M"} task="search_index_techdocs"
  • search info Added DefaultTechDocsCollatorFactory collator factory for type techdocs

Q: What is the expected behavior after the TechDocs Collator is initiated and running if it was not initiated and not running and the documentation is not indexed to AWS OpenSearch?

A: After the TechDocs Collator is initiated and running, we expect to see the documentation indexed to AWS OpenSearch.

Q: What are the steps to verify that the TechDocs Collator is initiated and running and the documentation is indexed to AWS OpenSearch if it was not initiated and not running and the documentation is not indexed to AWS OpenSearch?

A: To verify that the TechDocs Collator is initiated and running and the documentation is indexed to AWS OpenSearch, follow these steps:

  1. Go to the Search page
  2. Type a keyword and search
  3. Choose Documentation
  4. The page should come up with the documentation indexed to AWS OpenSearch.