An alternative to the suggestion above would be to provide the button in this deemphasized form and fill it out on hover:
In this form, we can keep the add button left of the clear button. Keep in mind that in both solutions, the button will be present on the DOM, leaving it easily accessible for CSS edits. In the earlier suggestion, a user could easily decide to keep the button present at all times once they have learn that the filter menu can be accessed there.