When connecting to Xero for the first time, you'll be asked for your connection details and also give an option to choose to import your Xero clients at the same time. Make sure this option is selected and then activate the integration.
You will be notified as soon as the import has been completed in the notification center.