It is definitely a fault that needs rectifying, especially if it has just been installed. The part of the flue that is exposed is the air duct, so assuming the exhaust, internal, section is correctly installed it won't create an immediate danger. However, an 'at risk' certification would suggest you do not use the appliance until the fault has been rectified.
If you are struggling to get them to return contact Gas Safe and ask them to inspect the installation as you have concerns. They will make the installer sort any defects found or risk losing their registration.