Support Symlinks For Developing

Introduction

As a developer, having a low-friction workflow is crucial for productivity and efficiency. When working on a plugin for Opnsense, it's essential to have a seamless development experience. In this article, we'll discuss the importance of supporting symlinks for developing and propose a solution to address the current limitations.

The Problem

When developing a plugin, it's common to want to work on a live system without having to constantly run sync commands. One approach to achieve this is by symlinking content from a git repository into the /usr/ directory. However, this approach is not currently supported in Opnsense, leading to issues with templating.

The Issue with Templating

When trying to reload the template, the following error occurs:

# sudo service configd restart
Stopping configd...done
Starting configd.

# sudo configctl template reload OPNsense/OpenApi
ERR

This is because the symlinked template folder is not picked up by os.walk in Template.list_modules().

The Solution

To address this issue, we propose adding the followlinks=True argument in template.py. This will allow the script to follow symlinks and pick up the template folder correctly.

Here's an example of how this can be achieved:

import os

# ...

def list_modules():
    for root, dirs, files in os.walk('/usr/local/opnsense/etc/opnsense/templates', followlinks=True):
        # ...

This change will enable developers to use symlinks for developing and make the workflow more efficient.

Alternatives Considered

Before proposing this solution, we considered alternative approaches:

• Rsync files from git into /usr/: While this approach can work, it's prone to errors and can lead to overwriting important files.
• Hard links: Hard links can be difficult to distinguish from regular files, making it challenging to clean up.
• Put the entire /usr/local/opnsense into git and unfuck it in the repo before PR: This approach requires extra work and can be cumbersome.
• Build opnsense from source: While building from source can be a viable option, it's not necessary for this use case.

Additional Context

To better understand the feature request, here are some additional context and screenshots:

• Contributing guidelines: Before adding a new report, please acknowledge the contributing guidelines at https://github.com/opnsense/core/blob/master/CONTRIBUTING.md.
• Issue checking: Please ensure that your issue is new by checking both open and closed issues at https://github.com/opnsense/core/issues?q=is%3Aissue.
• Symlink script: The symlink script used in this example can be found at https://github.com/opnsense/core/compare/master...fsackur:opnsense_core:symlinks?expand=1.

Conclusion

Supporting symlinks for developing is crucial for a low-friction workflow. By adding the followlinks=True argument in template.py, we can enable developers to use symlinks and make the development process more efficient. We propose this solution and invite the community to discuss and review the changes.

Introduction

In our previous article, we discussed the importance of supporting symlinks for developing and proposed a solution to address the current limitations. In this Q&A article, we'll answer some common questions and provide additional context to help clarify the feature request.

Q: What is the purpose of supporting symlinks for developing?

A: The primary purpose of supporting symlinks for developing is to enable developers to work on a live system without having to constantly run sync commands. This can significantly improve the development workflow and reduce the time spent on mundane tasks.

Q: How does the current implementation of Opnsense handle symlinks?

A: Currently, Opnsense does not handle symlinks correctly, leading to issues with templating. When trying to reload the template, the script fails to pick up the symlinked template folder.

Q: What is the proposed solution to address the issue?

A: We propose adding the followlinks=True argument in template.py. This will allow the script to follow symlinks and pick up the template folder correctly.

Q: Why is the followlinks=True argument necessary?

A: The followlinks=True argument is necessary because the current implementation of os.walk does not follow symlinks by default. By setting this argument to True, we can ensure that the script correctly handles symlinks and picks up the template folder.

Q: What are the benefits of supporting symlinks for developing?

A: Supporting symlinks for developing can bring several benefits, including:

• Improved development workflow: By enabling developers to work on a live system without having to constantly run sync commands, we can improve the development workflow and reduce the time spent on mundane tasks.
• Increased productivity: With a more efficient development workflow, developers can focus on writing code and improving the plugin, leading to increased productivity.
• Better collaboration: By supporting symlinks for developing, we can enable multiple developers to work on the same project simultaneously, leading to better collaboration and faster development.

Q: What are the potential risks or drawbacks of supporting symlinks for developing?

A: While supporting symlinks for developing can bring several benefits, there are also potential risks or drawbacks to consider, including:

• Security risks: If not implemented correctly, symlinks can pose a security risk, allowing malicious users to access sensitive data.
• Complexity: Supporting symlinks for developing can add complexity to the development workflow, requiring additional configuration and maintenance.
• Compatibility issues: Symlinks may not work correctly with all plugins or configurations, leading to compatibility issues.

Q: How can I contribute to the development of Opnsense and support symlinks for developing?

A: If you're interested in contributing to the development of Opnsense and supporting symlinks for developing, here are some steps you can take:

• Review the contributing guidelines: Before contributing to the project, please review the contributing guidelines at https://github.com/opnsense/core/blob/master/CONTRIBUTING.md.
• Check for existing issues: Before opening a new issue, please check for existing issues at https://github.com/opnsense/core/issues?q=is%3Aissue.
• Propose a solution: If you have a proposed solution to address the issue, please open a pull request with your changes.

Conclusion

Supporting symlinks for developing is a crucial feature for improving the development workflow and increasing productivity. By answering common questions and providing additional context, we hope to clarify the feature request and encourage the community to contribute to the development of Opnsense.