本人以下闲置域名转让,有意向请直接点下方链接查看购买
aliyun 一口价搜索并购买
(有意向,可议价,如议价过程中被第三人一口价先拍下,先抱歉恕不继续)
2020-02-17-Demo之iOS使用CoreData
Demo之iOS使用CoreData
准备
Xcode 添加 Coredata.framework 依赖库
确认是否存在 xxxCoreData.xcdatamodeld , 没有就新建一个, 在AppDelegate里可能会新出现一些代码
打开 xxxCoreData.xcdatamodeld 确认以下
ENTITIES里添加xxxEntity(也就是类obj),并添加属性及属性类型
CONFIGURATIONS里确认Entity及Class是否存在上一行的那个xxxEntity,其abstract不勾选(Demo用不上)
代码
1 | #import <CoreData/CoreData.h> |
以下代码是xcode直接生成在AppDelegate里的,不需要重复添加,列在这里只是备个份
1 | #pragma mark - Core Data stack |
2020-02-15-AFNetworking添加SecurityPolicy解决上传文件时https请求Status Code 200(正常)Request failed unacceptable content-type问题
使用 https://github.com/AFNetworking/AFNetworking 上传文件时https请求Status Code 200(正常)Request failed unacceptable content-type问题
需要在原代码中加入以下codeblock来解决
最终代码1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52AFHTTPSessionManager *manager = [AFHTTPSessionManager manager];
manager.requestSerializer= [AFHTTPRequestSerializer serializer];
//往下 2020.1.14更新此codeblock
//添加securityPolicy
AFSecurityPolicy *securityPolicy = [AFSecurityPolicy defaultPolicy];
securityPolicy.validatesDomainName = NO;
securityPolicy.allowInvalidCertificates = YES;
manager.securityPolicy = securityPolicy;
//需要请求前面加上定制化的配置
manager.responseSerializer.acceptableContentTypes= [NSSet setWithObjects:@"application/json", @"text/json", @"text/javascript",@"text/html", nil];
//往上 2020.1.14更新此codeblock
NSDictionary *dict = @{@"key":@"id111111",@"type":@"file4what"};
NSMutableData *data = [NSMutableData data];
NSMutableData *fileData = [NSMutableData dataWithContentsOfFile:@"filePath"];
NSString *spaceString1 = @"\n\n---------------------------------------------------------1---------------------------------------------------------\n\n";
[data appendData:[spaceString1 dataUsingEncoding:NSUTF8StringEncoding]];
[data appendData:fileData];
NSString *spaceString2 = @"\n\n---------------------------------------------------------split line---------------------------------------------------------\n\n";
[data appendData:[spaceString2 dataUsingEncoding:NSUTF8StringEncoding]];
// 在网络开发中,上传文件时,是文件不允许被覆盖,文件重名
// 要解决此问题,
// 可以在上传时使用当前的系统事件作为文件名
NSDateFormatter *formatter = [[NSDateFormatter alloc] init];
// 设置时间格式
formatter.dateFormat = @"yyyyMMddHHmmss";
NSString *str = [formatter stringFromDate:[NSDate date]];
NSString *fileName = [NSString stringWithFormat:@"abc%@.log", str];
[manager POST:@"https://www.xxx.com/app/upload.php" parameters:dict constructingBodyWithBlock:^(id<AFMultipartFormData> _Nonnull formData) {
//上传
/*
此方法参数
1. 要上传的[二进制数据]
2. 对应网站上[upload.php中]处理文件的[字段"Filedata"]
3. 要保存在服务器上的[文件名]
4. 上传文件的[mimeType]
*/
[formData appendPartWithFileData:data name:@"Filedata" fileName:fileName mimeType:@"text/html"];
} progress:^(NSProgress * _Nonnull uploadProgress) {
NSLog(@"%f",1.0 * uploadProgress.completedUnitCount / uploadProgress.totalUnitCount);
} success:^(NSURLSessionDataTask * _Nonnull task, id _Nullable responseObject) {
NSLog(@"上传日志成功 %@", responseObject);
//返回json
NSDictionary *jsonDic = responseObject;
//OK了开始处理
int ret = [jsonDic[@"code"] intValue];
if (ret == 0) {
NSString *xxx = jsonDic[@"xxx"];
//......
}
} failure:^(NSURLSessionDataTask * _Nullable task, NSError * _Nonnull error) {
NSLog(@"上传日志失败 %@", error);
}];
参考链接
https://www.jianshu.com/p/3bc4243e4c05
https://github.com/AFNetworking/AFNetworking
2020-02-13-Scrapy爬虫Demo_抓取出版社2020年春季学期使用教科书
(有删减)
源头及原因
http://xxx.com/jc/
出版社2020年春季学期使用教科书,方便在家自主学习。所以下载下来本地随时使用
Scrapy介绍
Scrapy is an application framework for crawling web sites and extracting structured data which can be used for a wide range of useful applications, like data mining, information processing or historical archival.
Scrapy安装
https://doc.scrapy.org/en/latest/intro/overview.html
爬虫创建及添加
创建
scrapy startproject jiaocai
生成爬虫
cd jiaocai
scrapy genspider jiaocai2020 http://xxx.com/jc
修改 jiaocai/spiders/jiaocai2020.py 文件内容,如下:
# -*- coding: utf-8 -*-
# Usage 使用方法: scrapy crawl jiaocai2020 -o xxx.csv
# //img//@data-src' 图片类似于这样的 <img data-src='http://xxxx/images/2018.jpg'> in start_urls
# //input//@data-src' 图片类似于这样的 <input data-src='http://xxxx/images/2018.jpg' type='image'> in start_urls
# 因为考虑反爬虫的问题,建议URL地址已抓出来后,直接将所有地址放到迅雷里打包下载
import scrapy
from scrapy import Request
class Jiaocai2020Spider(scrapy.Spider):
name = 'jiaocai2020'
# allowed_domains = ['http://xxx.com']
start_urls = ['http://xxx.com/jc/']
def parse(self, response):
print (response.url)
# for quote1 in response.xpath('//a/@href').extract():
# <li class="fl"><a href="./ywjygjkcjc/xdjc/" target="_blank">教科书</a></li>
for quote1 in response.xpath('//li[@class="fl"]//a/@href').extract():
# href="./ywjygjkcjc/xxdfjsys/"
ur11 = "http://xxx.com/jc/" + quote1[2:]
print (ur11)
# 'http://xxx.com/jc/zzwhjc/zzjyghjsys/',
yield scrapy.Request(ur11, callback=self.calbak, meta=None)
# yield scrapy.Request(ur11, callback=self.calbak, meta={'ur11': ur11})
def calbak(self, response):
print (response.url)
# kkp = response.meta['ur11']
# print (kkp)
# if "error" in response.body:
# self.logger.error("error1r")
# return
# <a href="./202002/P020200210281142511137.pdf" target="_blank" title="下载" class="btn_type_dl">下载</a>
for quote2 in response.xpath('//a[@class="btn_type_dl"]/@href').extract():
ur12 = response.url + quote2[2:]
# http://xxx.com/jc/zzwhjc/zzjyghjsys/202002/P020200211707082058148.pdf
print (ur12)
# 加 -o 参数时导出到文件时,会用到
yield {
'url' : ur12
}
# (直接下载图片到当前目录中,反爬严格的,可能会无法下载下来)
# 因为考虑反爬虫的问题,建议URL地址已抓出来后,直接将所有地址放到迅雷里打包下载
# if quote:
# filename = os.path.basename(quote)
# print filename
# filepath=os.path.join("/Users/xxx/Desktop",filename)
# urllib.urlretrieve(quote,filepath)
# 这个注释掉,需要时再用
# yield scrapy.Request(quote)
使用
Usage 使用方法: scrapy crawl jiaocai2020 -o xxx.csv
一些注意事项
1, 因为考虑反爬虫的问题,建议URL地址已抓出来后,直接将所有地址放到迅雷里打包下载
2, Scrapy爬虫出现Forbidden by robots.txt
关闭scrapy自带的ROBOTSTXT_OBEY功能,在setting找到这个变量,设置为False即可解决
ROBOTSTXT_OBEY = False
3, Settings.py文件的一些修改项
USER_AGENT = "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_8; en-us) AppleWebKit/534.50 (KHTML, like Gecko) Version/5.1 Safari/534.50"
ROBOTSTXT_OBEY = False
CONCURRENT_REQUESTS = 1
DOWNLOAD_DELAY = 1
DEFAULT_REQUEST_HEADERS = {
'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',
'Accept-Language': 'en',
}
AUTOTHROTTLE_START_DELAY = 5
AUTOTHROTTLE_MAX_DELAY = 60
创建 main.py 自动化执行 (未完成)
创建 main.py 自动化执行, 点击运行即可爬数据, 文件放在项目根目录(即scrapy.cfg所在目录)
main.py代码如下
import scrapy
from scrapy.cmdline import execute
import os,sys
sys.path.append(os.path.dirname(os.path.basename(__file__))) #注意,是爬虫名,不是项目名
execute(['scrapy','crawl','jiaocai2020'])
# from scrapy import cmdline
# cmdline.execute("scrapy crawl jiaocai2020".split())
参考链
https://www.jianshu.com/p/8d353c7cf606
https://zhuanlan.zhihu.com/p/32458936
2020-02-10-howto如何使用漂逸算法及边改iOS的swift语言Struct式的Framework边调App
howto如何使用漂逸算法及边改iOS的swift语言Struct式的Framework边调App
Transform coordinate between earth(WGS-84) and mars in china(GCJ-02).
火星坐标:我国自己加了飘逸搞的一套加密坐标 还原了飘逸算法
usage
1 | import HameToolsFramework |
Framework code
1 | import Foundation |