Configure each attribute and upload the image, select color and enter the label text you want to display on the frontend according to the display type.
To configure the attribute options, you need to first click on the "Configure Terms" link for the specific attribute and then click on the Edit link for the attribute term you want to configure.
When you land on the attribute term edit screen, you will find the section for the Product Attribute Options just at the bottom of the edit screen. Here, you can upload the attribute image, the color and also enter the label you want to display for the attribute on the frontend according to the display type. Here's a quick view of the options -