how to clipping UI texture？ now I want to clipping a square texture to a circle one
I believe that there are currently no circular clip options. But there are possibly some options that are useful for you.
If you are OK to blank out images you can overlay rings / squares (for example to make a telescopic sight mask)
You can use the new Media Library to upload images that better display what you want. This includes png format with transparency. So this could include an image of a circle with the outside of the circle transparent.
Whilst some of the UI textures are transparent themselves, you will get a lot more freedom of UI look if you use the Media Library to upload static images for the bulk of your UI design and supplement these with conventional UI components where required for interaction.