Wily diff output

[cometchaser]

Hi,

I wanted to use wily as an external tool in Pycharm, in order to see the wily diff of a file.
wily.exe diff foobar.py
However this fails and I get this message:

... File "c:\programdata\anaconda3\envs\foobar\lib\encodings\cp1252.py", line 19, in encode return codecs.charmap_encode(input,self.errors,encoding_table)[0] UnicodeEncodeError: 'charmap' codec can't encode characters in position 0-123: character maps to <undefined>

Same happens in the cmd when I wants to pipe the output to a file.
wily.exe diff foobar.py > wilydiff.txt

The output options are not valid for the diffs.
Any idea? Thanks

[tonybaloney]

Yikes!

What version of Python is this and what version of Wily do you have installed?

Please can you run wily index and share some of the output

[cometchaser]

Here see the conda environment file:
wilyenv.txt

and the index gives:
`$wily index
--------Configuration---------
Path: D:\Projects\foobar
Archiver: git
Operators: {'halstead', 'maintainability', 'cyclomatic', 'raw'}

-----------History------------
╒════════════╤═══════════════════════════════════╤════════════╕
│ Revision │ Author │ Date │
╞════════════╪═══════════════════════════════════╪════════════╡
│ 6a0dcb9 │ CometChaser │ 2019-03-22 │
├────────────┼───────────────────────────────────┼────────────┤
`
... and so on ...

I also add some more output from the wily diff:
`$ wily.exe diff foobar.py > test.txt

Using default metrics ['cyclomatic.complexity', 'maintainability.mi', 'raw.loc', 'halstead.h1']
Traceback (most recent call last):
File "c:\programdata\anaconda3\envs\foobar\lib\runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "c:\programdata\anaconda3\envs\foobar\lib\runpy.py", line 85, in run_code
exec(code, run_globals)
File "C:\ProgramData\Anaconda3\envs\foobar\Scripts\wily.exe_main.py", line 9, in
File "c:\programdata\anaconda3\envs\foobar\lib\site-packages\click\core.py", line 764, in call
return self.main(*args, **kwargs)
File "c:\programdata\anaconda3\envs\foobar\lib\site-packages\click\core.py", line 717, in main
rv = self.invoke(ctx)
File "c:\programdata\anaconda3\envs\foobar\lib\site-packages\click\core.py", line 1137, in invoke
return process_result(sub_ctx.command.invoke(sub_ctx))
File "c:\programdata\anaconda3\envs\foobar\lib\site-packages\click\core.py", line 956, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "c:\programdata\anaconda3\envs\foobar\lib\site-packages\click\core.py", line 555, in invoke
return callback(*args, **kwargs)
File "c:\programdata\anaconda3\envs\foobar\lib\site-packages\click\decorators.py", line 17, in new_func
return f(get_current_context(), *args, **kwargs)
File "c:\programdata\anaconda3\envs\foobar\lib\site-packages\wily_main.py", line 245, in diff
config=config, files=files, metrics=metrics, changes_only=not all, detail=detail
File "c:\programdata\anaconda3\envs\foobar\lib\site-packages\wily\commands\diff.py", line 130, in diff
headers=headers, tabular_data=results, tablefmt=DEFAULT_GRID_STYLE
File "c:\programdata\anaconda3\envs\foobar\lib\site-packages\colorama\ansitowin32.py", line 41, in write
self.__convertor.write(text)
File "c:\programdata\anaconda3\envs\foobar\lib\site-packages\colorama\ansitowin32.py", line 162, in write
self.write_and_convert(text)
File "c:\programdata\anaconda3\envs\foobar\lib\site-packages\colorama\ansitowin32.py", line 187, in write_and_convert
self.write_plain_text(text, cursor, start)
File "c:\programdata\anaconda3\envs\foobar\lib\site-packages\colorama\ansitowin32.py", line 195, in write_plain_text
self.wrapped.write(text[start:end])
File "c:\programdata\anaconda3\envs\foobar\lib\site-packages\colorama\ansitowin32.py", line 41, in write
self.__convertor.write(text)
File "c:\programdata\anaconda3\envs\foobar\lib\site-packages\colorama\ansitowin32.py", line 162, in write
self.write_and_convert(text)
File "c:\programdata\anaconda3\envs\foobar\lib\site-packages\colorama\ansitowin32.py", line 190, in write_and_convert
self.write_plain_text(text, cursor, len(text))
File "c:\programdata\anaconda3\envs\foobar\lib\site-packages\colorama\ansitowin32.py", line 195, in write_plain_text
self.wrapped.write(text[start:end])
File "c:\programdata\anaconda3\envs\foobar\lib\encodings\cp1252.py", line 19, in encode
return codecs.charmap_encode(input,self.errors,encoding_table)[0]
UnicodeEncodeError: 'charmap' codec can't encode characters in position 0-145: character maps to
`

[GabdaZM]

I have a similar problem, but with almost every command (build command works):
wily-err-log.txt

[tonybaloney]

@GabdaZM are you also using anaconda? I think this is related to the default encoding profiles in the environment.

[GabdaZM]

No, I am using PyCharm on wndows, and using UTF-8 encoding, LF line ending.

[cbetylkata]

I am having the same issue when I tried using the pre-commit hook. Also using PyCharm on Windowns 10 with UTF-8 encoding and LF as line separator.
wily, version 1.19.0

Has there been any resolution or possible workaround for this issue?

P.S. Manually typed in diff command works OK, but pre-commit hook doesn`t.

[devdanzin]

Seems to be the generic case for the issue I found in #182: the Windows terminal (or default file encoding) may not be able to handle all characters that we want to print/write. We should be able to encode and decode the output with an error handler. However, we'd first need to choose which error handler (or whether to make it configurable) to use.

[devdanzin]

There's a different issue going on here: the DEFAULT_GRID_STYLE is "fancy_grid", which uses characters outside of what charmap can understand.

It's possible to check sys.stdout.encoding and, if it's not UTF-8, use "grid" to tabulate. It gives comparable output with ASCII-only characters. I'll send a PR so we can discuss whether it's worth it and how to do it.

The other issue, what happens when some data (like author names or commit messages) we want to print contain unprintable characters, should be solvable by using errors="replace" in a encode/decode operation.